2024-09-10T22:24:39.477 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T22:24:39.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:24:39.594 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/openfiletable} email: hyelloji@redhat.com first_in_suite: false job_id: '7899736' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: syntax: v2 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1Fs+8Boynk8kJGfLjLLwdY9hg5X+tNhRxfQISpr4Y470RckpsCHdMKhqH7W1gUc/OMZAdOX/vAQcRskGmxjlI= smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDawvmFXo93shfOjb0E/aN8gdeMXJN0l3772KVopvMRMs6qqyJEZHk75tnPFQiOYYqCTFLvVyNccMx6I9pLcAI8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_openfiletable teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T22:24:39.594 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T22:24:39.595 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T22:24:39.595 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T22:24:39.596 INFO:teuthology.task.internal:Checking packages... 2024-09-10T22:24:39.618 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T22:24:39.618 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T22:24:39.618 INFO:teuthology.packaging:ref: None 2024-09-10T22:24:39.618 INFO:teuthology.packaging:tag: None 2024-09-10T22:24:39.618 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:24:39.618 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:24:39.618 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:24:39.769 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T22:24:39.771 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T22:24:39.776 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T22:24:39.776 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T22:24:39.792 INFO:teuthology.task.internal:Saving configuration 2024-09-10T22:24:39.808 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T22:24:39.813 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T22:24:39.847 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:19:01.743022', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1Fs+8Boynk8kJGfLjLLwdY9hg5X+tNhRxfQISpr4Y470RckpsCHdMKhqH7W1gUc/OMZAdOX/vAQcRskGmxjlI='} 2024-09-10T22:24:39.879 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 22:19:01.744479', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDawvmFXo93shfOjb0E/aN8gdeMXJN0l3772KVopvMRMs6qqyJEZHk75tnPFQiOYYqCTFLvVyNccMx6I9pLcAI8='} 2024-09-10T22:24:39.879 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T22:24:39.884 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T22:24:39.884 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T22:24:39.885 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T22:24:39.959 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4c97ae7be0>, signals=[15]) 2024-09-10T22:24:39.959 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T22:24:39.964 INFO:teuthology.task.internal:Opening connections... 2024-09-10T22:24:39.965 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:24:39.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:24:40.053 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:24:40.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:24:40.136 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T22:24:40.142 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-09-10T22:24:40.161 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-09-10T22:24:40.162 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-09-10T22:24:40.220 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-09-10T22:24:40.220 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:24:40.221 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:24:40.222 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-09-10T22:24:40.257 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-09-10T22:24:40.276 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-09-10T22:24:40.276 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-09-10T22:24:40.333 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T22:24:40.334 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T22:24:40.335 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T22:24:40.335 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-09-10T22:24:40.366 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T22:24:40.374 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T22:24:40.451 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T22:24:40.451 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:24:40.454 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-09-10T22:24:40.471 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T22:24:40.476 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T22:24:40.476 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:24:40.512 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-09-10T22:24:40.680 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T22:24:40.748 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T22:24:40.762 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T22:24:40.762 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-10T22:24:40.762 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-10T22:24:40.762 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:24:40.763 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T22:24:40.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:24:40.780 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-10T22:24:40.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:24:40.781 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-10T22:24:40.838 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-503.el9.x86_64 2024-09-10T22:24:40.838 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-503.el9.x86_64 2024-09-10T22:24:40.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-09-10T22:24:40.840 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-503.el9.x86_64 2024-09-10T22:24:40.841 INFO:teuthology.task.kernel:Running kernel on smithi066: 5.14.0-503.el9.x86_64 2024-09-10T22:24:40.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum install -y kernel 2024-09-10T22:24:44.134 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.2 MB 00:01 2024-09-10T22:24:44.410 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-09-10T22:24:45.943 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-10T22:24:48.270 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 20 MB 00:02 2024-09-10T22:24:52.361 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-10T22:24:53.303 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.5 MB 00:01 2024-09-10T22:24:54.036 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-10T22:24:54.762 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-10T22:24:55.582 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-10T22:24:56.249 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T22:25:01.046 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-10T22:25:02.113 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T22:25:03.069 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T22:25:03.069 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T22:25:03.163 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-10T22:25:03.169 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-10T22:25:03.169 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-09-10T22:25:03.169 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-10T22:25:03.169 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-10T22:25:03.169 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-09-10T22:25:03.170 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:25:03.171 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 79 M 2024-09-10T22:25:03.171 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 126 M 2024-09-10T22:25:03.171 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-10T22:25:03.281 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 268 kB/s | 23 kB 00:00 2024-09-10T22:25:04.228 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T22:25:04.229 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T22:25:04.316 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-09-10T22:25:04.325 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T22:25:04.325 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-09-10T22:25:04.325 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T22:25:04.325 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout:Install 4 Packages 2024-09-10T22:25:04.326 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:25:04.327 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 79 M 2024-09-10T22:25:04.327 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 126 M 2024-09-10T22:25:04.327 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-09-10T22:25:04.940 INFO:teuthology.orchestra.run.smithi066.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 39 kB/s | 23 kB 00:00 2024-09-10T22:25:05.581 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-09-10T22:25:07.789 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 7.5 MB/s | 35 MB 00:04 2024-09-10T22:25:09.354 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.7 MB/s | 29 MB 00:06 2024-09-10T22:25:09.358 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:25:09.358 INFO:teuthology.orchestra.run.smithi094.stdout:Total 13 MB/s | 79 MB 00:06 2024-09-10T22:25:10.041 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-10T22:25:10.388 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-10T22:25:10.388 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-10T22:25:11.349 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-10T22:25:11.349 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-10T22:25:13.456 INFO:teuthology.orchestra.run.smithi066.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.8 MB/s | 35 MB 00:09 2024-09-10T22:25:13.779 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-10T22:25:13.889 INFO:teuthology.orchestra.run.smithi066.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.6 MB/s | 16 MB 00:09 2024-09-10T22:25:14.231 INFO:teuthology.orchestra.run.smithi066.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.1 MB/s | 29 MB 00:09 2024-09-10T22:25:14.232 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:25:14.232 INFO:teuthology.orchestra.run.smithi066.stdout:Total 8.0 MB/s | 79 MB 00:09 2024-09-10T22:25:14.512 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:25:14.606 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:25:14.948 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-09-10T22:25:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-09-10T22:25:15.348 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-09-10T22:25:16.355 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-09-10T22:25:16.356 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-09-10T22:25:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:25:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:25:18.962 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-09-10T22:25:19.650 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:25:19.732 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:25:22.384 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:25:22.423 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:25:22.658 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:25:22.783 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:25:27.312 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:25:27.991 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:25:28.034 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:25:33.273 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:05.837 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:05.876 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:26:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:26:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:26:07.117 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:07.117 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:26:07.117 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-10T22:26:07.117 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T22:26:07.117 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T22:26:07.118 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T22:26:07.118 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T22:26:07.118 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:26:07.118 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-10T22:26:07.519 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:26:07.594 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:26:07.594 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:26:07.595 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T22:26:07.595 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T22:26:07.595 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T22:26:07.595 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-10T22:26:07.595 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T22:26:07.595 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:26:07.654 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:26:07.654 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:26:07.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-09-10T22:26:07.686 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-09-10T22:26:07.686 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-09-10T22:26:07.686 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-09-10T22:26:07.686 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 82.el9 2024-09-10T22:26:07.686 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-09-10T22:26:07.686 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T22:26:07.686 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8274670 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T22:26:07.687 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T22:26:07.688 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T22:26:07.688 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-09-10T22:26:07.688 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:26:07.688 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T22:26:07.689 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T22:26:07.689 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 5.14.0-505.el9.x86_64 2024-09-10T22:26:07.689 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T22:26:08.307 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-10T22:26:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T22:26:09.682 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-10T22:26:09.685 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-10T22:26:09.700 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.qnTRZu4p3l 2024-09-10T22:26:09.701 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qnTRZu4p3l 2024-09-10T22:26:09.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.qnTRZu4p3l 2024-09-10T22:26:09.897 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.qnTRZu4p3l is 6KB 2024-09-10T22:26:09.946 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.qnTRZu4p3l 2024-09-10T22:26:09.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T22:26:10.031 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T22:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T22:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T22:26:10.032 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T22:26:10.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T22:26:10.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-09-10T22:26:13.668 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:13.711 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:14.081 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:14.082 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T22:26:14.082 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T22:26:14.082 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T22:26:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T22:26:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:26:14.953 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-09-10T22:26:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T22:26:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T22:26:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T22:26:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T22:26:14.954 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:26:14.954 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-09-10T22:26:15.303 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:26:15.382 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:26:15.382 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:26:15.382 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T22:26:15.382 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T22:26:15.382 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T22:26:15.382 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T22:26:15.382 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi066.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T22:26:15.382 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T22:26:15.443 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T22:26:15.443 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T22:26:15.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -qi grub2-tools 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Name : grub2-tools 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Epoch : 1 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Version : 2.06 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Release : 82.el9 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: x86_64 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Group : Unspecified 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Size : 8274670 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:License : GPLv3+ 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T22:26:15.476 INFO:teuthology.orchestra.run.smithi066.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:Packager : builder@centos.org 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:Vendor : CentOS 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:Summary : Support tools for GRUB. 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:Description : 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T22:26:15.477 INFO:teuthology.orchestra.run.smithi066.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T22:26:15.478 INFO:teuthology.orchestra.run.smithi066.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T22:26:15.478 INFO:teuthology.orchestra.run.smithi066.stdout:hardware devices. 2024-09-10T22:26:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:26:15.478 INFO:teuthology.orchestra.run.smithi066.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T22:26:15.479 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T22:26:15.479 INFO:teuthology.task.kernel:Updating grub on smithi066 to boot 5.14.0-505.el9.x86_64 2024-09-10T22:26:15.479 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T22:26:16.137 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-09-10T22:26:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T22:26:17.454 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-09-10T22:26:17.462 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-09-10T22:26:17.479 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.aX7EsODXZi 2024-09-10T22:26:17.479 DEBUG:teuthology.orchestra.run.smithi066:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aX7EsODXZi 2024-09-10T22:26:17.546 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.aX7EsODXZi 2024-09-10T22:26:17.688 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.aX7EsODXZi is 6KB 2024-09-10T22:26:17.737 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.aX7EsODXZi 2024-09-10T22:26:17.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T22:26:17.820 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T22:26:17.821 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T22:26:17.821 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T22:26:17.821 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T22:26:17.822 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T22:26:17.970 DEBUG:teuthology.orchestra.run.smithi066:> sudo shutdown -r now 2024-09-10T22:26:40.213 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T22:26:40.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T22:26:40.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:26:47.981 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T22:26:47.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:26:47.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:26:58.749 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-10T22:27:06.429 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T22:27:07.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T22:27:07.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:27:15.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:27:15.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:27:18.529 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T22:27:30.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:27:30.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:27:33.601 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T22:27:48.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:27:48.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:28:07.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T22:28:07.762 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T22:28:19.764 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T22:28:19.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:28:20.103 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-10T22:28:20.440 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T22:28:20.440 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-505.el9.x86_64"... 2024-09-10T22:28:20.441 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-10T22:28:20.457 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-505.el9.x86_64 2024-09-10T22:28:20.457 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T22:28:20.457 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T22:28:20.458 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T22:28:21.459 DEBUG:teuthology.parallel:result is None 2024-09-10T22:28:25.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:28:25.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:28:25.378 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-10T22:28:25.902 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:28:25.902 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T22:28:25.902 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-10T22:28:25.931 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-505.el9.x86_64 2024-09-10T22:28:25.932 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T22:28:25.932 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T22:28:25.932 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T22:28:26.933 DEBUG:teuthology.parallel:result is None 2024-09-10T22:28:26.933 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T22:28:26.940 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T22:28:26.940 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:28:26.942 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T22:28:26.960 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T22:28:26.967 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T22:28:26.977 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T22:28:26.978 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:28:26.998 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T22:28:27.220 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T22:28:27.226 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T22:28:27.226 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:28:27.229 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T22:28:27.313 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:28:27.330 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:28:27.588 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:28:27.676 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T22:28:27.678 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T22:28:27.685 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T22:28:27.686 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:28:27.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T22:28:27.718 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T22:28:27.727 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T22:28:27.728 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:28:27.758 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T22:28:27.778 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:28:27.857 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:28:27.910 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:28:28.000 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:28:28.034 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:28:28.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:28:28.099 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:28:28.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T22:28:28.205 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:28:28.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T22:28:28.333 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:28:28.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T22:28:28.401 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-09-10T22:28:28.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-09-10T22:28:28.443 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:28:28.488 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:28:28.960 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T22:28:28.970 INFO:teuthology.task.internal:Starting timer... 2024-09-10T22:28:28.971 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T22:28:29.007 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T22:28:29.015 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-09-10T22:28:29.061 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-09-10T22:28:29.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-09-10T22:28:29.124 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-09-10T22:28:29.126 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T22:28:29.126 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-09-10T22:28:29.149 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-09-10T22:28:29.149 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-09-10T22:28:29.176 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-09-10T22:28:29.177 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi094.front.sepia.ceph.com': 'permissive'} 2024-09-10T22:28:29.177 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T22:28:29.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:28:29.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T22:28:29.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:28:29.245 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T22:28:29.245 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:28:29.286 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-09-10T22:28:29.323 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T22:28:29.332 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T22:28:29.335 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T22:28:29.456 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T22:28:29.467 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T22:28:29.468 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi094.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T22:32:16.797 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi094.front.sepia.ceph.com')] 2024-09-10T22:32:16.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:32:16.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:32:16.886 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-10T22:32:16.974 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T22:32:16.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T22:32:16.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T22:32:17.055 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-10T22:32:17.135 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-10T22:32:17.136 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T22:32:17.146 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T22:32:17.146 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:32:17.147 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:32:17.149 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T22:32:17.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:32:17.178 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:32:17.193 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:32:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T22:32:17.218 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-09-10T22:32:17.219 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T22:32:17.230 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-09-10T22:32:17.242 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-09-10T22:32:17.245 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:32:17.253 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-09-10T22:32:17.259 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:32:17.267 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T22:32:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T22:32:17.313 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:32:17.316 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:32:17.317 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-09-10T22:32:17.317 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.317 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.317 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.317 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.338 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:32:17.341 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:32:17.341 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-09-10T22:32:17.341 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.341 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.341 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.342 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:32:17.342 INFO:teuthology.run_tasks:Running task install... 2024-09-10T22:32:17.350 DEBUG:teuthology.task.install:project ceph 2024-09-10T22:32:17.350 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T22:32:17.351 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T22:32:17.351 INFO:teuthology.task.install:Using flavor: default 2024-09-10T22:32:17.356 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T22:32:17.356 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T22:32:17.356 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:32:17.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:32:17.359 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T22:32:17.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:32:17.518 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:32:17.519 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:32:17.530 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T22:32:17.531 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T22:32:17.669 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:32:17.669 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:32:17.669 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:32:17.674 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T22:32:17.674 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:32:17.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T22:32:17.696 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:32:17.696 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:32:17.700 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T22:32:17.701 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T22:32:17.763 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:32:17.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T22:32:17.839 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:32:17.840 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T22:32:17.905 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-09-10T22:32:17.905 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2024-09-10T22:32:17.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-09-10T22:32:17.907 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2024-09-10T22:32:18.137 INFO:teuthology.orchestra.run.smithi094.stdout:45 files removed 2024-09-10T22:32:18.140 INFO:teuthology.orchestra.run.smithi066.stdout:45 files removed 2024-09-10T22:32:18.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:32:18.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T22:32:18.805 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 237 kB/s | 84 kB 00:00 2024-09-10T22:32:18.856 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 216 kB/s | 84 kB 00:00 2024-09-10T22:32:19.189 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 71 kB/s | 24 kB 00:00 2024-09-10T22:32:19.190 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 84 kB/s | 24 kB 00:00 2024-09-10T22:32:19.581 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-09-10T22:32:19.607 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-09-10T22:32:20.316 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-10T22:32:22.126 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-10T22:32:22.415 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-10T22:32:26.241 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 20 MB 00:02 2024-09-10T22:32:30.759 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-10T22:32:30.984 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.5 MB 00:04 2024-09-10T22:32:32.302 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-10T22:32:32.810 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-10T22:32:33.722 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T22:32:34.056 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-10T22:32:39.229 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T22:32:39.736 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T22:32:41.633 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:32:41.634 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:32:41.647 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:32:41.648 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:32:41.649 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:32:41.651 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:32:41.653 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:32:41.655 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:32:41.656 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:32:41.721 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-10T22:32:41.742 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-09-10T22:32:41.742 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-10T22:32:41.742 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-09-10T22:32:41.742 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-10T22:32:41.742 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:32:41.742 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:32:41.743 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:32:41.744 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:32:41.745 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:32:41.746 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:32:41.747 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:32:41.748 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:32:41.749 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:32:41.750 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:32:41.751 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:32:41.752 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:32:41.753 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-09-10T22:32:41.754 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:32:41.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:32:41.755 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-10T22:32:41.755 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-09-10T22:32:41.755 INFO:teuthology.orchestra.run.smithi094.stdout:Install 149 Packages 2024-09-10T22:32:41.755 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 8 Packages 2024-09-10T22:32:41.755 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:32:41.755 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 240 M 2024-09-10T22:32:41.756 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-10T22:32:42.097 INFO:teuthology.orchestra.run.smithi094.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-09-10T22:32:42.198 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:32:42.198 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T22:32:42.210 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T22:32:42.211 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T22:32:42.212 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T22:32:42.213 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T22:32:42.214 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T22:32:42.217 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T22:32:42.218 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T22:32:42.284 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-09-10T22:32:42.288 INFO:teuthology.orchestra.run.smithi094.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 4.6 MB/s | 900 kB 00:00 2024-09-10T22:32:42.304 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-09-10T22:32:42.304 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2024-09-10T22:32:42.304 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-09-10T22:32:42.304 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-09-10T22:32:42.304 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T22:32:42.305 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T22:32:42.306 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T22:32:42.307 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T22:32:42.308 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T22:32:42.309 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T22:32:42.310 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T22:32:42.311 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T22:32:42.312 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T22:32:42.313 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T22:32:42.314 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T22:32:42.315 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout:Installing weak dependencies: 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:32:42.316 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-09-10T22:32:42.317 INFO:teuthology.orchestra.run.smithi066.stdout:======================================================================================= 2024-09-10T22:32:42.317 INFO:teuthology.orchestra.run.smithi066.stdout:Install 149 Packages 2024-09-10T22:32:42.317 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 8 Packages 2024-09-10T22:32:42.317 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:32:42.317 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 240 M 2024-09-10T22:32:42.317 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-09-10T22:32:42.381 INFO:teuthology.orchestra.run.smithi094.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.5 MB/s | 145 kB 00:00 2024-09-10T22:32:42.473 INFO:teuthology.orchestra.run.smithi094.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-10T22:32:42.573 INFO:teuthology.orchestra.run.smithi094.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 15 MB/s | 1.5 MB 00:00 2024-09-10T22:32:42.623 INFO:teuthology.orchestra.run.smithi094.stdout:(6/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 9.8 MB/s | 2.4 MB 00:00 2024-09-10T22:32:42.840 INFO:teuthology.orchestra.run.smithi094.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 18 MB/s | 4.8 MB 00:00 2024-09-10T22:32:42.846 INFO:teuthology.orchestra.run.smithi066.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-09-10T22:32:43.113 INFO:teuthology.orchestra.run.smithi066.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 3.3 MB/s | 900 kB 00:00 2024-09-10T22:32:43.149 INFO:teuthology.orchestra.run.smithi094.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 18 MB/s | 22 MB 00:01 2024-09-10T22:32:43.163 INFO:teuthology.orchestra.run.smithi066.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 2.8 MB/s | 145 kB 00:00 2024-09-10T22:32:43.191 INFO:teuthology.orchestra.run.smithi094.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 608 kB/s | 25 kB 00:00 2024-09-10T22:32:43.538 INFO:teuthology.orchestra.run.smithi066.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 6.4 MB/s | 2.4 MB 00:00 2024-09-10T22:32:43.541 INFO:teuthology.orchestra.run.smithi094.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 19 MB/s | 17 MB 00:00 2024-09-10T22:32:43.600 INFO:teuthology.orchestra.run.smithi094.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 3.8 MB/s | 225 kB 00:00 2024-09-10T22:32:43.675 INFO:teuthology.orchestra.run.smithi094.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 361 kB/s | 27 kB 00:00 2024-09-10T22:32:43.750 INFO:teuthology.orchestra.run.smithi094.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 424 kB/s | 32 kB 00:00 2024-09-10T22:32:43.755 INFO:teuthology.orchestra.run.smithi066.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-09-10T22:32:43.842 INFO:teuthology.orchestra.run.smithi094.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.1 MB/s | 755 kB 00:00 2024-09-10T22:32:43.926 INFO:teuthology.orchestra.run.smithi094.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.9 MB/s | 162 kB 00:00 2024-09-10T22:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 1.5 MB/s | 127 kB 00:00 2024-09-10T22:32:44.110 INFO:teuthology.orchestra.run.smithi094.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 5.0 MB/s | 507 kB 00:00 2024-09-10T22:32:44.172 INFO:teuthology.orchestra.run.smithi066.stdout:(6/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 3.5 MB/s | 5.5 MB 00:01 2024-09-10T22:32:44.439 INFO:teuthology.orchestra.run.smithi066.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 7.0 MB/s | 4.8 MB 00:00 2024-09-10T22:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 16 MB/s | 5.5 MB 00:00 2024-09-10T22:32:44.535 INFO:teuthology.orchestra.run.smithi094.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 602 kB/s | 45 kB 00:00 2024-09-10T22:32:44.610 INFO:teuthology.orchestra.run.smithi094.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.9 MB/s | 143 kB 00:00 2024-09-10T22:32:44.686 INFO:teuthology.orchestra.run.smithi094.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.1 MB/s | 161 kB 00:00 2024-09-10T22:32:44.803 INFO:teuthology.orchestra.run.smithi094.stdout:(22/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 5.6 MB/s | 11 MB 00:01 2024-09-10T22:32:44.828 INFO:teuthology.orchestra.run.smithi094.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 2.2 MB/s | 321 kB 00:00 2024-09-10T22:32:44.862 INFO:teuthology.orchestra.run.smithi094.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T22:32:44.887 INFO:teuthology.orchestra.run.smithi094.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.7 MB/s | 99 kB 00:00 2024-09-10T22:32:44.929 INFO:teuthology.orchestra.run.smithi094.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.2 MB/s | 85 kB 00:00 2024-09-10T22:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 1.8 MB/s | 171 kB 00:00 2024-09-10T22:32:45.155 INFO:teuthology.orchestra.run.smithi094.stdout:(28/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 12 MB/s | 3.2 MB 00:00 2024-09-10T22:32:45.180 INFO:teuthology.orchestra.run.smithi094.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 215 kB/s | 34 kB 00:00 2024-09-10T22:32:45.205 INFO:teuthology.orchestra.run.smithi094.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.0 MB/s | 154 kB 00:00 2024-09-10T22:32:45.373 INFO:teuthology.orchestra.run.smithi066.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 12 MB/s | 11 MB 00:00 2024-09-10T22:32:45.415 INFO:teuthology.orchestra.run.smithi066.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 605 kB/s | 25 kB 00:00 2024-09-10T22:32:45.664 INFO:teuthology.orchestra.run.smithi094.stdout:(31/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 20 MB/s | 50 MB 00:02 2024-09-10T22:32:45.722 INFO:teuthology.orchestra.run.smithi094.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.9 MB/s | 291 kB 00:00 2024-09-10T22:32:45.798 INFO:teuthology.orchestra.run.smithi094.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 657 kB/s | 49 kB 00:00 2024-09-10T22:32:45.933 INFO:teuthology.orchestra.run.smithi094.stdout:(34/157): ceph-mgr-diskprediction-local-19.3.0- 10 MB/s | 7.4 MB 00:00 2024-09-10T22:32:45.957 INFO:teuthology.orchestra.run.smithi094.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 104 kB/s | 16 kB 00:00 2024-09-10T22:32:45.990 INFO:teuthology.orchestra.run.smithi094.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.8 MB/s | 285 kB 00:00 2024-09-10T22:32:46.066 INFO:teuthology.orchestra.run.smithi094.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 7.0 MB/s | 783 kB 00:00 2024-09-10T22:32:46.091 INFO:teuthology.orchestra.run.smithi094.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 182 kB/s | 18 kB 00:00 2024-09-10T22:32:46.225 INFO:teuthology.orchestra.run.smithi094.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-10T22:32:46.317 INFO:teuthology.orchestra.run.smithi094.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-10T22:32:46.350 INFO:teuthology.orchestra.run.smithi094.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.0 MB/s | 801 kB 00:00 2024-09-10T22:32:46.384 INFO:teuthology.orchestra.run.smithi094.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-10T22:32:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 7.7 MB/s | 17 MB 00:02 2024-09-10T22:32:46.434 INFO:teuthology.orchestra.run.smithi094.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-10T22:32:46.459 INFO:teuthology.orchestra.run.smithi094.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-10T22:32:46.465 INFO:teuthology.orchestra.run.smithi066.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-10T22:32:46.485 INFO:teuthology.orchestra.run.smithi094.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T22:32:46.510 INFO:teuthology.orchestra.run.smithi094.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T22:32:46.515 INFO:teuthology.orchestra.run.smithi066.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 542 kB/s | 27 kB 00:00 2024-09-10T22:32:46.535 INFO:teuthology.orchestra.run.smithi094.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T22:32:46.565 INFO:teuthology.orchestra.run.smithi066.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 636 kB/s | 32 kB 00:00 2024-09-10T22:32:46.577 INFO:teuthology.orchestra.run.smithi094.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-09-10T22:32:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-09-10T22:32:46.636 INFO:teuthology.orchestra.run.smithi094.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-10T22:32:46.674 INFO:teuthology.orchestra.run.smithi066.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 6.8 MB/s | 755 kB 00:00 2024-09-10T22:32:46.724 INFO:teuthology.orchestra.run.smithi066.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 3.2 MB/s | 162 kB 00:00 2024-09-10T22:32:46.770 INFO:teuthology.orchestra.run.smithi094.stdout:(51/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 8.9 MB/s | 14 MB 00:01 2024-09-10T22:32:46.774 INFO:teuthology.orchestra.run.smithi066.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T22:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 124 kB/s | 22 kB 00:00 2024-09-10T22:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 1.6 MB/s | 313 kB 00:00 2024-09-10T22:32:46.858 INFO:teuthology.orchestra.run.smithi066.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 5.9 MB/s | 507 kB 00:00 2024-09-10T22:32:46.887 INFO:teuthology.orchestra.run.smithi094.stdout:(54/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 32 MB/s | 3.0 MB 00:00 2024-09-10T22:32:46.913 INFO:teuthology.orchestra.run.smithi094.stdout:(55/157): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-09-10T22:32:46.938 INFO:teuthology.orchestra.run.smithi094.stdout:(56/157): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-09-10T22:32:46.963 INFO:teuthology.orchestra.run.smithi094.stdout:(57/157): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-10T22:32:46.997 INFO:teuthology.orchestra.run.smithi094.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-10T22:32:47.022 INFO:teuthology.orchestra.run.smithi094.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-10T22:32:47.047 INFO:teuthology.orchestra.run.smithi094.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-10T22:32:47.081 INFO:teuthology.orchestra.run.smithi094.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-10T22:32:47.115 INFO:teuthology.orchestra.run.smithi094.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-10T22:32:47.148 INFO:teuthology.orchestra.run.smithi094.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-10T22:32:47.159 INFO:teuthology.orchestra.run.smithi066.stdout:(18/157): ceph-common-19.3.0-4854.g8f219c47.el9 4.8 MB/s | 22 MB 00:04 2024-09-10T22:32:47.174 INFO:teuthology.orchestra.run.smithi094.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-09-10T22:32:47.199 INFO:teuthology.orchestra.run.smithi094.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T22:32:47.209 INFO:teuthology.orchestra.run.smithi066.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 910 kB/s | 45 kB 00:00 2024-09-10T22:32:47.224 INFO:teuthology.orchestra.run.smithi094.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-10T22:32:47.249 INFO:teuthology.orchestra.run.smithi094.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-10T22:32:47.284 INFO:teuthology.orchestra.run.smithi066.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.9 MB/s | 143 kB 00:00 2024-09-10T22:32:47.333 INFO:teuthology.orchestra.run.smithi094.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T22:32:47.342 INFO:teuthology.orchestra.run.smithi066.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.7 MB/s | 161 kB 00:00 2024-09-10T22:32:47.409 INFO:teuthology.orchestra.run.smithi094.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 24 MB/s | 4.9 MB 00:00 2024-09-10T22:32:47.418 INFO:teuthology.orchestra.run.smithi066.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 4.2 MB/s | 321 kB 00:00 2024-09-10T22:32:47.450 INFO:teuthology.orchestra.run.smithi094.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-09-10T22:32:47.476 INFO:teuthology.orchestra.run.smithi094.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-10T22:32:47.509 INFO:teuthology.orchestra.run.smithi094.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-10T22:32:47.535 INFO:teuthology.orchestra.run.smithi066.stdout:(23/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 8.2 MB/s | 5.5 MB 00:00 2024-09-10T22:32:47.551 INFO:teuthology.orchestra.run.smithi094.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-10T22:32:47.560 INFO:teuthology.orchestra.run.smithi066.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 2.1 MB/s | 306 kB 00:00 2024-09-10T22:32:47.576 INFO:teuthology.orchestra.run.smithi094.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T22:32:47.586 INFO:teuthology.orchestra.run.smithi066.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.9 MB/s | 99 kB 00:00 2024-09-10T22:32:47.611 INFO:teuthology.orchestra.run.smithi066.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T22:32:47.618 INFO:teuthology.orchestra.run.smithi094.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T22:32:47.661 INFO:teuthology.orchestra.run.smithi066.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 3.3 MB/s | 171 kB 00:00 2024-09-10T22:32:47.694 INFO:teuthology.orchestra.run.smithi094.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-10T22:32:47.703 INFO:teuthology.orchestra.run.smithi066.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 829 kB/s | 34 kB 00:00 2024-09-10T22:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T22:32:47.744 INFO:teuthology.orchestra.run.smithi094.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-10T22:32:47.753 INFO:teuthology.orchestra.run.smithi066.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.0 MB/s | 154 kB 00:00 2024-09-10T22:32:47.769 INFO:teuthology.orchestra.run.smithi094.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-10T22:32:47.795 INFO:teuthology.orchestra.run.smithi094.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-10T22:32:47.828 INFO:teuthology.orchestra.run.smithi094.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-10T22:32:47.854 INFO:teuthology.orchestra.run.smithi094.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-10T22:32:47.879 INFO:teuthology.orchestra.run.smithi066.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 11 MB/s | 3.2 MB 00:00 2024-09-10T22:32:47.937 INFO:teuthology.orchestra.run.smithi094.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-10T22:32:47.971 INFO:teuthology.orchestra.run.smithi094.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-10T22:32:47.996 INFO:teuthology.orchestra.run.smithi094.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-10T22:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T22:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-10T22:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-10T22:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T22:32:48.123 INFO:teuthology.orchestra.run.smithi094.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T22:32:48.157 INFO:teuthology.orchestra.run.smithi094.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 869 kB/s | 51 kB 00:00 2024-09-10T22:32:48.182 INFO:teuthology.orchestra.run.smithi094.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 912 kB/s | 54 kB 00:00 2024-09-10T22:32:48.266 INFO:teuthology.orchestra.run.smithi094.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-10T22:32:48.291 INFO:teuthology.orchestra.run.smithi094.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-10T22:32:48.316 INFO:teuthology.orchestra.run.smithi094.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-10T22:32:48.329 INFO:teuthology.orchestra.run.smithi066.stdout:(31/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 17 MB/s | 50 MB 00:02 2024-09-10T22:32:48.350 INFO:teuthology.orchestra.run.smithi094.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T22:32:48.375 INFO:teuthology.orchestra.run.smithi094.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-10T22:32:48.401 INFO:teuthology.orchestra.run.smithi094.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-10T22:32:48.404 INFO:teuthology.orchestra.run.smithi066.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 3.8 MB/s | 291 kB 00:00 2024-09-10T22:32:48.434 INFO:teuthology.orchestra.run.smithi094.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-10T22:32:48.446 INFO:teuthology.orchestra.run.smithi066.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 1.2 MB/s | 49 kB 00:00 2024-09-10T22:32:48.459 INFO:teuthology.orchestra.run.smithi094.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-10T22:32:48.484 INFO:teuthology.orchestra.run.smithi094.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T22:32:48.488 INFO:teuthology.orchestra.run.smithi066.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 398 kB/s | 16 kB 00:00 2024-09-10T22:32:48.510 INFO:teuthology.orchestra.run.smithi094.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T22:32:48.538 INFO:teuthology.orchestra.run.smithi066.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.6 MB/s | 285 kB 00:00 2024-09-10T22:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-09-10T22:32:48.613 INFO:teuthology.orchestra.run.smithi066.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 10 MB/s | 783 kB 00:00 2024-09-10T22:32:48.660 INFO:teuthology.orchestra.run.smithi094.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-09-10T22:32:48.664 INFO:teuthology.orchestra.run.smithi066.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 365 kB/s | 18 kB 00:00 2024-09-10T22:32:48.747 INFO:teuthology.orchestra.run.smithi066.stdout:(38/157): ceph-mgr-diskprediction-local-19.3.0- 8.5 MB/s | 7.4 MB 00:00 2024-09-10T22:32:48.844 INFO:teuthology.orchestra.run.smithi094.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-10T22:32:48.847 INFO:teuthology.orchestra.run.smithi066.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 435 kB/s | 80 kB 00:00 2024-09-10T22:32:48.911 INFO:teuthology.orchestra.run.smithi094.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 5.1 MB/s | 309 kB 00:00 2024-09-10T22:32:48.931 INFO:teuthology.orchestra.run.smithi066.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-10T22:32:48.945 INFO:teuthology.orchestra.run.smithi094.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-09-10T22:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 331 kB/s | 19 kB 00:00 2024-09-10T22:32:48.990 INFO:teuthology.orchestra.run.smithi066.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-09-10T22:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-10T22:32:49.015 INFO:teuthology.orchestra.run.smithi066.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-10T22:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-09-10T22:32:49.062 INFO:teuthology.orchestra.run.smithi094.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T22:32:49.065 INFO:teuthology.orchestra.run.smithi066.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T22:32:49.096 INFO:teuthology.orchestra.run.smithi094.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-09-10T22:32:49.116 INFO:teuthology.orchestra.run.smithi066.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T22:32:49.121 INFO:teuthology.orchestra.run.smithi094.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-10T22:32:49.158 INFO:teuthology.orchestra.run.smithi066.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-09-10T22:32:49.183 INFO:teuthology.orchestra.run.smithi066.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T22:32:49.188 INFO:teuthology.orchestra.run.smithi094.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-10T22:32:49.208 INFO:teuthology.orchestra.run.smithi066.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T22:32:49.222 INFO:teuthology.orchestra.run.smithi094.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-10T22:32:49.248 INFO:teuthology.orchestra.run.smithi094.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T22:32:49.272 INFO:teuthology.orchestra.run.smithi094.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-10T22:32:49.298 INFO:teuthology.orchestra.run.smithi094.stdout:(118/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-10T22:32:49.331 INFO:teuthology.orchestra.run.smithi094.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-10T22:32:49.358 INFO:teuthology.orchestra.run.smithi066.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-10T22:32:49.373 INFO:teuthology.orchestra.run.smithi094.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-10T22:32:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-10T22:32:49.407 INFO:teuthology.orchestra.run.smithi094.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-10T22:32:49.409 INFO:teuthology.orchestra.run.smithi066.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-10T22:32:49.434 INFO:teuthology.orchestra.run.smithi066.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 519 kB/s | 22 kB 00:00 2024-09-10T22:32:49.441 INFO:teuthology.orchestra.run.smithi094.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-10T22:32:49.459 INFO:teuthology.orchestra.run.smithi066.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-10T22:32:49.483 INFO:teuthology.orchestra.run.smithi094.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-10T22:32:49.484 INFO:teuthology.orchestra.run.smithi066.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-10T22:32:49.508 INFO:teuthology.orchestra.run.smithi094.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-10T22:32:49.575 INFO:teuthology.orchestra.run.smithi094.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-09-10T22:32:49.600 INFO:teuthology.orchestra.run.smithi094.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-09-10T22:32:49.626 INFO:teuthology.orchestra.run.smithi094.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-10T22:32:49.635 INFO:teuthology.orchestra.run.smithi066.stdout:(54/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 7.5 MB/s | 14 MB 00:01 2024-09-10T22:32:49.651 INFO:teuthology.orchestra.run.smithi094.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-10T22:32:49.661 INFO:teuthology.orchestra.run.smithi066.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 104 kB/s | 18 kB 00:00 2024-09-10T22:32:49.676 INFO:teuthology.orchestra.run.smithi094.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-10T22:32:49.693 INFO:teuthology.orchestra.run.smithi094.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-10T22:32:49.719 INFO:teuthology.orchestra.run.smithi066.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-10T22:32:49.727 INFO:teuthology.orchestra.run.smithi094.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T22:32:49.744 INFO:teuthology.orchestra.run.smithi066.stdout:(57/157): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-10T22:32:49.770 INFO:teuthology.orchestra.run.smithi066.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-10T22:32:49.777 INFO:teuthology.orchestra.run.smithi094.stdout:(132/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-09-10T22:32:49.794 INFO:teuthology.orchestra.run.smithi094.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-09-10T22:32:49.795 INFO:teuthology.orchestra.run.smithi066.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-10T22:32:49.819 INFO:teuthology.orchestra.run.smithi094.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-10T22:32:49.820 INFO:teuthology.orchestra.run.smithi066.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-09-10T22:32:49.845 INFO:teuthology.orchestra.run.smithi094.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-10T22:32:49.862 INFO:teuthology.orchestra.run.smithi066.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-10T22:32:49.870 INFO:teuthology.orchestra.run.smithi094.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-10T22:32:49.879 INFO:teuthology.orchestra.run.smithi066.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-10T22:32:49.895 INFO:teuthology.orchestra.run.smithi094.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T22:32:49.912 INFO:teuthology.orchestra.run.smithi066.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-10T22:32:49.920 INFO:teuthology.orchestra.run.smithi094.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-10T22:32:49.929 INFO:teuthology.orchestra.run.smithi066.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-09-10T22:32:49.954 INFO:teuthology.orchestra.run.smithi094.stdout:(139/157): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-09-10T22:32:49.955 INFO:teuthology.orchestra.run.smithi066.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-10T22:32:49.971 INFO:teuthology.orchestra.run.smithi066.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 921 kB/s | 38 kB 00:00 2024-09-10T22:32:49.979 INFO:teuthology.orchestra.run.smithi094.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-10T22:32:49.997 INFO:teuthology.orchestra.run.smithi066.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T22:32:50.005 INFO:teuthology.orchestra.run.smithi094.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T22:32:50.030 INFO:teuthology.orchestra.run.smithi094.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T22:32:50.064 INFO:teuthology.orchestra.run.smithi094.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-09-10T22:32:50.089 INFO:teuthology.orchestra.run.smithi094.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-10T22:32:50.089 INFO:teuthology.orchestra.run.smithi066.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 37 MB/s | 4.9 MB 00:00 2024-09-10T22:32:50.122 INFO:teuthology.orchestra.run.smithi094.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-10T22:32:50.147 INFO:teuthology.orchestra.run.smithi066.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-10T22:32:50.148 INFO:teuthology.orchestra.run.smithi094.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-10T22:32:50.173 INFO:teuthology.orchestra.run.smithi094.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T22:32:50.189 INFO:teuthology.orchestra.run.smithi066.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-09-10T22:32:50.207 INFO:teuthology.orchestra.run.smithi094.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-10T22:32:50.214 INFO:teuthology.orchestra.run.smithi066.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-10T22:32:50.240 INFO:teuthology.orchestra.run.smithi066.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-10T22:32:50.282 INFO:teuthology.orchestra.run.smithi066.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-10T22:32:50.307 INFO:teuthology.orchestra.run.smithi066.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T22:32:50.332 INFO:teuthology.orchestra.run.smithi094.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-10T22:32:50.349 INFO:teuthology.orchestra.run.smithi066.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T22:32:50.416 INFO:teuthology.orchestra.run.smithi094.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 28 MB/s | 2.3 MB 00:00 2024-09-10T22:32:50.424 INFO:teuthology.orchestra.run.smithi066.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-10T22:32:50.449 INFO:teuthology.orchestra.run.smithi066.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T22:32:50.475 INFO:teuthology.orchestra.run.smithi066.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-10T22:32:50.491 INFO:teuthology.orchestra.run.smithi094.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-10T22:32:50.500 INFO:teuthology.orchestra.run.smithi066.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-10T22:32:50.537 INFO:teuthology.orchestra.run.smithi066.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-10T22:32:50.550 INFO:teuthology.orchestra.run.smithi094.stdout:(152/157): librados2-19.3.0-4854.g8f219c47.el9. 9.2 MB/s | 3.5 MB 00:00 2024-09-10T22:32:50.550 INFO:teuthology.orchestra.run.smithi066.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T22:32:50.575 INFO:teuthology.orchestra.run.smithi094.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 488 kB/s | 41 kB 00:00 2024-09-10T22:32:50.576 INFO:teuthology.orchestra.run.smithi066.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-10T22:32:50.600 INFO:teuthology.orchestra.run.smithi094.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 4.8 MB/s | 249 kB 00:00 2024-09-10T22:32:50.651 INFO:teuthology.orchestra.run.smithi066.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-10T22:32:50.667 INFO:teuthology.orchestra.run.smithi094.stdout:(155/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 7.0 MB/s | 3.2 MB 00:00 2024-09-10T22:32:50.676 INFO:teuthology.orchestra.run.smithi066.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-10T22:32:50.701 INFO:teuthology.orchestra.run.smithi094.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-10T22:32:50.701 INFO:teuthology.orchestra.run.smithi066.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T22:32:50.735 INFO:teuthology.orchestra.run.smithi066.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T22:32:50.759 INFO:teuthology.orchestra.run.smithi094.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-10T22:32:50.761 INFO:teuthology.orchestra.run.smithi066.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-09-10T22:32:50.766 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:32:50.766 INFO:teuthology.orchestra.run.smithi094.stdout:Total 27 MB/s | 240 MB 00:09 2024-09-10T22:32:50.786 INFO:teuthology.orchestra.run.smithi066.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T22:32:50.811 INFO:teuthology.orchestra.run.smithi066.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T22:32:50.837 INFO:teuthology.orchestra.run.smithi066.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T22:32:50.862 INFO:teuthology.orchestra.run.smithi066.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T22:32:50.887 INFO:teuthology.orchestra.run.smithi066.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T22:32:50.929 INFO:teuthology.orchestra.run.smithi066.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T22:32:50.996 INFO:teuthology.orchestra.run.smithi066.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-10T22:32:51.022 INFO:teuthology.orchestra.run.smithi066.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-09-10T22:32:51.047 INFO:teuthology.orchestra.run.smithi066.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T22:32:51.072 INFO:teuthology.orchestra.run.smithi066.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-10T22:32:51.097 INFO:teuthology.orchestra.run.smithi066.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T22:32:51.122 INFO:teuthology.orchestra.run.smithi066.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T22:32:51.147 INFO:teuthology.orchestra.run.smithi066.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T22:32:51.173 INFO:teuthology.orchestra.run.smithi066.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T22:32:51.198 INFO:teuthology.orchestra.run.smithi066.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T22:32:51.223 INFO:teuthology.orchestra.run.smithi066.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-10T22:32:51.449 INFO:teuthology.orchestra.run.smithi066.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-10T22:32:51.474 INFO:teuthology.orchestra.run.smithi066.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-09-10T22:32:51.549 INFO:teuthology.orchestra.run.smithi066.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-09-10T22:32:51.574 INFO:teuthology.orchestra.run.smithi066.stdout:(107/157): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-09-10T22:32:51.591 INFO:teuthology.orchestra.run.smithi066.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 466 kB/s | 19 kB 00:00 2024-09-10T22:32:51.609 INFO:teuthology.orchestra.run.smithi066.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-10T22:32:51.642 INFO:teuthology.orchestra.run.smithi066.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-10T22:32:51.667 INFO:teuthology.orchestra.run.smithi066.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-09-10T22:32:51.700 INFO:teuthology.orchestra.run.smithi066.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-10T22:32:51.726 INFO:teuthology.orchestra.run.smithi066.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-10T22:32:51.809 INFO:teuthology.orchestra.run.smithi066.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-10T22:32:51.843 INFO:teuthology.orchestra.run.smithi066.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-10T22:32:51.859 INFO:teuthology.orchestra.run.smithi066.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-10T22:32:51.885 INFO:teuthology.orchestra.run.smithi066.stdout:(117/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-09-10T22:32:51.911 INFO:teuthology.orchestra.run.smithi066.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 279 kB/s | 14 kB 00:00 2024-09-10T22:32:51.936 INFO:teuthology.orchestra.run.smithi066.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T22:32:51.961 INFO:teuthology.orchestra.run.smithi066.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T22:32:51.973 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-10T22:32:51.986 INFO:teuthology.orchestra.run.smithi066.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T22:32:52.012 INFO:teuthology.orchestra.run.smithi066.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T22:32:52.070 INFO:teuthology.orchestra.run.smithi066.stdout:(123/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-09-10T22:32:52.095 INFO:teuthology.orchestra.run.smithi066.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-10T22:32:52.121 INFO:teuthology.orchestra.run.smithi066.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-10T22:32:52.127 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-10T22:32:52.127 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-10T22:32:52.146 INFO:teuthology.orchestra.run.smithi066.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-10T22:32:52.171 INFO:teuthology.orchestra.run.smithi066.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-10T22:32:52.196 INFO:teuthology.orchestra.run.smithi066.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-10T22:32:52.222 INFO:teuthology.orchestra.run.smithi066.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T22:32:52.247 INFO:teuthology.orchestra.run.smithi066.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T22:32:52.281 INFO:teuthology.orchestra.run.smithi066.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T22:32:52.314 INFO:teuthology.orchestra.run.smithi066.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-10T22:32:52.364 INFO:teuthology.orchestra.run.smithi066.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-10T22:32:52.390 INFO:teuthology.orchestra.run.smithi066.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-10T22:32:52.415 INFO:teuthology.orchestra.run.smithi066.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-09-10T22:32:52.457 INFO:teuthology.orchestra.run.smithi066.stdout:(136/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-09-10T22:32:52.482 INFO:teuthology.orchestra.run.smithi066.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-09-10T22:32:52.507 INFO:teuthology.orchestra.run.smithi066.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-09-10T22:32:52.533 INFO:teuthology.orchestra.run.smithi066.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-10T22:32:52.558 INFO:teuthology.orchestra.run.smithi066.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T22:32:52.583 INFO:teuthology.orchestra.run.smithi066.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-10T22:32:52.609 INFO:teuthology.orchestra.run.smithi066.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T22:32:52.634 INFO:teuthology.orchestra.run.smithi066.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-10T22:32:52.659 INFO:teuthology.orchestra.run.smithi066.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T22:32:52.684 INFO:teuthology.orchestra.run.smithi066.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-10T22:32:52.709 INFO:teuthology.orchestra.run.smithi066.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-10T22:32:52.735 INFO:teuthology.orchestra.run.smithi066.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T22:32:52.768 INFO:teuthology.orchestra.run.smithi066.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T22:32:52.827 INFO:teuthology.orchestra.run.smithi066.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T22:32:53.086 INFO:teuthology.orchestra.run.smithi066.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 9.9 MB/s | 3.5 MB 00:00 2024-09-10T22:32:53.194 INFO:teuthology.orchestra.run.smithi066.stdout:(151/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 7.6 MB/s | 3.2 MB 00:00 2024-09-10T22:32:53.280 INFO:teuthology.orchestra.run.smithi066.stdout:(152/157): NetworkManager-1.49.90-1.el9.x86_64. 5.2 MB/s | 2.3 MB 00:00 2024-09-10T22:32:53.295 INFO:teuthology.orchestra.run.smithi066.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 407 kB/s | 41 kB 00:00 2024-09-10T22:32:53.345 INFO:teuthology.orchestra.run.smithi066.stdout:(154/157): NetworkManager-libnm-1.49.90-1.el9.x 7.0 MB/s | 1.8 MB 00:00 2024-09-10T22:32:53.370 INFO:teuthology.orchestra.run.smithi066.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 2.6 MB/s | 249 kB 00:00 2024-09-10T22:32:53.637 INFO:teuthology.orchestra.run.smithi066.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 1.4 MB 00:00 2024-09-10T22:32:53.687 INFO:teuthology.orchestra.run.smithi066.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-09-10T22:32:53.693 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-09-10T22:32:53.694 INFO:teuthology.orchestra.run.smithi066.stdout:Total 21 MB/s | 240 MB 00:11 2024-09-10T22:32:54.485 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-10T22:32:54.485 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-10T22:32:54.978 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-09-10T22:32:55.148 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-09-10T22:32:55.148 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-09-10T22:32:57.532 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-09-10T22:32:57.532 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-09-10T22:33:00.866 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-10T22:33:01.006 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:33:01.054 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:33:01.115 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:33:01.147 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:33:01.173 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:33:01.230 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:33:01.559 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:33:01.691 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:33:02.146 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:33:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:33:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:33:02.404 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:33:02.461 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:33:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:33:02.654 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:33:02.675 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:33:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:33:02.864 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:33:03.172 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:33:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:33:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:33:03.571 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:33:03.674 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:33:03.709 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:33:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:33:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:33:03.868 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:33:03.901 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:33:03.944 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:33:03.978 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:33:04.005 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:33:04.073 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:33:04.127 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:33:04.168 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:33:04.210 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:33:04.286 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:33:04.292 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-09-10T22:33:04.321 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:33:04.364 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:33:04.383 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T22:33:04.393 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:33:04.431 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T22:33:04.462 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:33:04.484 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T22:33:04.517 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:33:04.526 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T22:33:04.559 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T22:33:04.598 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:33:04.619 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T22:33:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:33:04.785 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:33:04.972 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T22:33:05.068 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:33:05.193 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:33:05.228 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:33:05.303 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:33:05.321 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:33:05.349 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:33:05.379 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T22:33:05.380 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:33:05.411 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:33:05.454 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:33:05.492 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:33:05.543 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:33:05.561 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T22:33:05.622 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T22:33:05.649 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:33:05.713 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:33:05.756 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:33:05.762 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:33:05.784 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:33:05.847 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T22:33:05.860 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:33:05.878 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:33:05.902 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T22:33:05.925 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:33:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:33:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:33:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:33:06.073 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:33:06.105 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:33:06.141 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:33:06.184 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:33:06.226 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:33:06.237 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T22:33:06.359 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T22:33:06.368 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:33:06.412 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:33:06.453 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:33:06.466 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:33:06.527 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:33:06.554 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:33:06.588 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:33:06.628 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:33:06.653 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:33:06.678 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:33:06.763 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T22:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:33:06.879 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T22:33:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:33:06.907 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T22:33:06.942 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T22:33:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:33:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:33:07.019 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T22:33:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:33:07.069 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T22:33:07.078 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:33:07.109 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T22:33:07.110 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:33:07.152 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T22:33:07.182 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:33:07.184 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T22:33:07.212 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T22:33:07.283 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T22:33:07.343 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T22:33:07.350 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:33:07.373 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T22:33:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:33:07.417 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T22:33:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:33:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:33:07.495 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T22:33:07.527 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T22:33:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:33:07.571 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T22:33:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:33:07.599 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T22:33:07.629 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:33:07.682 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T22:33:07.731 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:33:07.731 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T22:33:07.796 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:33:07.809 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T22:33:07.963 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T22:33:08.018 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T22:33:08.503 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:33:08.565 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:33:08.572 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T22:33:08.598 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:33:08.664 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T22:33:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:33:08.708 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T22:33:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:33:08.751 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T22:33:08.789 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T22:33:08.790 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:33:08.831 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T22:33:08.877 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:33:08.918 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:33:08.991 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T22:33:09.067 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T22:33:09.103 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:33:09.133 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:33:09.162 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T22:33:09.210 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:33:09.224 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T22:33:09.297 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T22:33:09.335 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T22:33:09.371 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T22:33:09.419 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T22:33:09.462 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T22:33:09.514 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T22:33:09.564 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T22:33:09.708 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T22:33:09.749 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T22:33:09.791 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T22:33:09.866 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:33:09.898 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T22:33:09.989 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:33:10.078 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T22:33:10.376 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T22:33:10.415 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T22:33:10.439 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T22:33:10.548 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T22:33:10.691 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T22:33:10.746 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T22:33:10.775 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T22:33:10.816 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T22:33:10.856 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T22:33:10.896 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T22:33:10.926 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T22:33:10.960 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T22:33:10.993 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T22:33:11.035 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T22:33:11.078 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T22:33:11.168 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T22:33:11.226 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T22:33:11.233 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:33:11.256 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T22:33:11.327 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:33:11.355 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:33:11.363 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T22:33:11.388 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:33:11.416 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T22:33:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:33:11.768 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:33:11.850 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:33:11.881 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:33:11.917 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:33:12.180 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T22:33:12.224 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T22:33:12.257 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T22:33:12.361 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T22:33:12.413 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T22:33:12.418 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:33:12.458 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T22:33:12.462 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:33:12.517 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:33:12.547 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:33:12.549 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T22:33:12.585 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T22:33:12.776 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T22:33:12.807 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T22:33:12.888 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T22:33:13.584 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T22:33:13.701 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T22:33:14.973 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:33:15.002 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:33:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T22:33:15.101 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T22:33:15.129 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T22:33:15.163 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T22:33:15.503 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T22:33:15.542 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:33:15.572 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:33:15.606 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:33:15.634 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T22:33:15.664 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T22:33:15.700 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T22:33:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:33:15.703 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:33:15.703 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:33:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:15.741 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:33:16.229 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T22:33:16.261 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:33:16.325 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T22:33:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T22:33:18.828 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:33:18.868 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:33:19.469 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T22:33:19.504 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:33:19.588 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T22:33:19.589 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T22:33:19.589 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T22:33:19.589 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:19.630 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-09-10T22:33:27.790 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:33:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:33:27.982 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:27.982 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:33:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:33:29.527 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:33:29.527 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:29.527 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:33:29.528 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:33:29.528 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:33:29.528 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:29.751 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:33:29.774 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:33:29.774 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:29.774 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:33:29.774 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:33:29.774 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:33:29.774 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:33:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:33:30.250 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:30.250 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:33:30.250 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:33:30.250 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:33:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:30.349 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:33:30.414 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:33:30.469 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:33:30.687 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:33:30.736 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:33:30.806 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:33:30.895 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:33:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:33:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:33:31.111 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:33:31.138 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:33:31.543 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:33:31.571 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:33:31.611 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:33:31.677 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:33:31.702 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:33:31.996 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T22:33:31.996 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2024-09-10T22:33:31.997 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2024-09-10T22:33:31.997 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2024-09-10T22:33:31.997 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2024-09-10T22:33:31.997 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2024-09-10T22:33:31.997 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /root 2024-09-10T22:33:31.997 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2024-09-10T22:33:31.997 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:32.181 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:33:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T22:33:32.191 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:32.191 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T22:33:32.191 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:33.760 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:33:33.795 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T22:33:33.795 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:33.795 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T22:33:33.795 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:33:33.796 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T22:33:33.796 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:34.028 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:33:34.053 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T22:33:34.053 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:34.054 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T22:33:34.054 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:33:34.054 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T22:33:34.054 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:34.528 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:33:34.554 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T22:33:34.554 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:34.554 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T22:33:34.554 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:33:34.554 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T22:33:34.554 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:34.668 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T22:33:34.734 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T22:33:34.798 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T22:33:34.867 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:33:34.926 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:33:35.003 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T22:33:35.055 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T22:33:35.153 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T22:33:35.250 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T22:33:35.402 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T22:33:35.447 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T22:33:35.478 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T22:33:35.887 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:33:35.917 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:33:35.928 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:33:35.954 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T22:33:35.986 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T22:33:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:33:36.056 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T22:33:36.085 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:33:36.189 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:33:36.219 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:33:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:33:36.244 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:36.244 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:33:36.244 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:33:36.244 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:33:36.244 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:33:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:33:36.415 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:33:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:33:37.423 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:33:37.456 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:33:37.480 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:33:37.480 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:37.480 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:33:37.480 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:33:37.480 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:33:37.480 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:37.524 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:33:37.547 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:33:37.547 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:37.547 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:33:37.547 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:37.591 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:33:37.614 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:33:37.614 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:37.614 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:33:37.614 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:33:37.614 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:33:37.615 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:37.925 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:33:37.948 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:33:37.948 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:37.948 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:33:37.949 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:33:37.949 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:33:37.949 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:39.463 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T22:33:39.518 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:33:40.536 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T22:33:40.568 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:33:40.690 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T22:33:40.862 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T22:33:40.895 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:33:40.921 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T22:33:40.921 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:40.921 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T22:33:40.921 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:33:40.921 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T22:33:40.922 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:40.970 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:33:40.983 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T22:33:41.085 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T22:33:41.149 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T22:33:42.160 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T22:33:42.206 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:33:42.232 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T22:33:42.232 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:42.232 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T22:33:42.232 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:33:42.232 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T22:33:42.233 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:42.273 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:33:42.298 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T22:33:42.298 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:42.298 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T22:33:42.298 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:42.349 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:33:42.375 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T22:33:42.375 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:42.375 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T22:33:42.375 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:33:42.376 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T22:33:42.376 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:42.646 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:33:42.687 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:33:42.708 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:33:42.728 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:33:42.734 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T22:33:42.734 INFO:teuthology.orchestra.run.smithi066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T22:33:42.734 INFO:teuthology.orchestra.run.smithi066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T22:33:42.734 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:33:42.734 INFO:teuthology.orchestra.run.smithi066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T22:33:42.734 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:42.760 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:33:42.788 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:33:42.982 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:33:43.034 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:33:43.065 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:33:43.094 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:33:43.141 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:33:43.756 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:33:43.798 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:33:43.818 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:33:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:33:44.736 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:33:44.788 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:33:44.883 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:33:44.915 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:33:44.940 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:33:44.966 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:33:44.966 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:33:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:33:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:33:45.126 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:33:45.151 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:33:45.156 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:33:45.156 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:33:45.328 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:33:45.328 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:33:45.420 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:33:45.485 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:33:47.712 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T22:33:47.753 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T22:33:47.795 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T22:33:47.826 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T22:33:47.853 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T22:33:48.062 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T22:33:48.108 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T22:33:48.138 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T22:33:48.168 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T22:33:48.205 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T22:33:48.875 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:33:48.911 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:33:48.933 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T22:33:49.836 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T22:33:49.932 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T22:33:49.985 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T22:33:50.086 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T22:33:50.118 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T22:33:50.144 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T22:33:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T22:33:50.170 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:33:50.271 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T22:33:50.271 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:33:50.354 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T22:33:50.379 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T22:33:50.385 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:33:50.385 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:33:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T22:33:50.573 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:33:50.657 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T22:33:50.733 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:33:52.208 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:33:52.209 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:33:52.210 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:33:52.221 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:33:52.222 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:33:52.223 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:33:52.224 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:33:52.225 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:33:52.226 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:33:52.227 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:33:52.228 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:33:52.229 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:33:52.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:33:52.231 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:33:52.232 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:33:52.232 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:33:52.232 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:33:54.795 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:33:54.795 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:54.795 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-09-10T22:33:54.795 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:54.795 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:54.795 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.798 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.799 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:33:54.800 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:33:54.801 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:33:54.802 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:33:54.803 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:33:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:33:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:33:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:33:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:33:54.808 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-10T22:33:55.266 DEBUG:teuthology.parallel:result is None 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T22:33:56.770 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T22:33:56.771 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T22:33:56.772 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T22:33:56.772 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T22:33:56.772 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T22:33:56.772 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T22:33:56.772 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T22:33:56.772 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T22:33:56.772 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T22:33:56.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T22:33:56.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T22:33:56.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T22:33:56.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T22:33:56.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T22:33:56.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T22:33:56.784 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T22:33:56.785 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T22:33:56.786 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T22:33:56.787 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T22:33:56.788 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T22:33:56.789 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T22:33:56.790 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T22:33:56.791 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T22:33:56.792 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T22:33:56.793 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T22:33:56.794 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T22:33:56.794 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T22:33:56.794 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T22:33:56.794 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T22:33:56.794 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T22:33:59.270 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.271 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.272 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T22:33:59.273 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.274 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T22:33:59.275 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T22:33:59.276 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T22:33:59.277 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T22:33:59.278 INFO:teuthology.orchestra.run.smithi066.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.279 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T22:33:59.280 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T22:33:59.281 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T22:33:59.282 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T22:33:59.282 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T22:33:59.282 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:33:59.282 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-09-10T22:33:59.734 DEBUG:teuthology.parallel:result is None 2024-09-10T22:33:59.734 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:33:59.893 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:33:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:33:59.919 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:33:59.919 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:33:59.921 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:34:00.069 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T22:34:00.095 INFO:teuthology.orchestra.run.smithi094.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T22:34:00.096 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T22:34:00.096 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T22:34:00.098 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T22:34:00.098 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:00.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:34:00.129 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:00.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T22:34:00.167 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T22:34:00.167 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:00.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:34:00.199 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:34:00.268 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:00.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T22:34:00.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T22:34:00.363 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T22:34:00.363 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:00.363 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:34:00.393 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:34:00.462 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:00.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T22:34:00.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T22:34:00.556 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T22:34:00.557 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:00.557 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:34:00.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:34:00.654 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:00.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T22:34:00.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T22:34:00.747 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T22:34:00.829 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T22:34:00.829 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /var/log/ceph 2024-09-10T22:34:00.832 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-09-10T22:34:00.858 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T22:34:00.859 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T22:34:00.901 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T22:34:00.927 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T22:34:00.927 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T22:34:00.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T22:34:01.002 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T22:34:01.002 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T22:34:01.002 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T22:34:01.002 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:01.003 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T22:34:01.054 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T22:34:01.177 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T22:34:01.204 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:01.204 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T22:34:01.220 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T22:34:01.343 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T22:34:01.375 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T22:34:01.375 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T22:34:01.375 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/openfiletable}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899736', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1Fs+8Boynk8kJGfLjLLwdY9hg5X+tNhRxfQISpr4Y470RckpsCHdMKhqH7W1gUc/OMZAdOX/vAQcRskGmxjlI=', 'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDawvmFXo93shfOjb0E/aN8gdeMXJN0l3772KVopvMRMs6qqyJEZHk75tnPFQiOYYqCTFLvVyNccMx6I9pLcAI8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_openfiletable']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T22:34:01.376 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T22:34:01.378 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T22:34:01.424 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T22:34:01.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T22:34:01.491 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:01.491 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:34:01.516 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:34:01.517 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-09-10T22:34:01.573 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:34:01.573 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:01.573 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T22:34:01.573 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:01.573 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:01.573 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 22:33:51.106686610 +0000 2024-09-10T22:34:01.573 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 22:31:57.232448632 +0000 2024-09-10T22:34:01.574 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 22:31:57.232448632 +0000 2024-09-10T22:34:01.574 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 22:31:57.232448632 +0000 2024-09-10T22:34:01.574 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:34:01.637 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:34:01.638 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:34:01.638 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000256019 s, 2.0 MB/s 2024-09-10T22:34:01.639 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:34:01.695 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-09-10T22:34:01.751 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 22:33:51.106686610 +0000 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 22:31:57.570446300 +0000 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 22:31:57.570446300 +0000 2024-09-10T22:34:01.752 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 22:31:57.570446300 +0000 2024-09-10T22:34:01.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:34:01.816 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:34:01.816 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:34:01.816 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000189209 s, 2.7 MB/s 2024-09-10T22:34:01.817 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:34:01.874 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-09-10T22:34:01.929 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:34:01.929 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:01.930 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T22:34:01.930 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:01.930 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:01.930 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 22:33:51.107686603 +0000 2024-09-10T22:34:01.930 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 22:31:57.907443974 +0000 2024-09-10T22:34:01.930 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 22:31:57.907443974 +0000 2024-09-10T22:34:01.930 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 22:31:57.907443974 +0000 2024-09-10T22:34:01.930 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:34:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:34:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:34:01.994 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000245384 s, 2.1 MB/s 2024-09-10T22:34:01.995 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:34:02.053 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-09-10T22:34:02.108 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:34:02.108 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:02.109 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T22:34:02.109 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:02.109 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:02.109 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 22:33:51.107686603 +0000 2024-09-10T22:34:02.109 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 22:31:58.243441655 +0000 2024-09-10T22:34:02.109 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 22:31:58.243441655 +0000 2024-09-10T22:34:02.109 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 22:31:58.243441655 +0000 2024-09-10T22:34:02.109 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:34:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:34:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:34:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000202536 s, 2.5 MB/s 2024-09-10T22:34:02.174 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:34:02.232 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:02.232 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:02.232 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T22:34:02.252 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T22:34:02.252 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-09-10T22:34:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T22:34:02.310 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:02.311 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T22:34:02.311 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:02.311 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:02.311 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 22:33:45.852955832 +0000 2024-09-10T22:34:02.311 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 22:31:52.130423393 +0000 2024-09-10T22:34:02.311 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 22:31:52.130423393 +0000 2024-09-10T22:34:02.311 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 22:31:52.130423393 +0000 2024-09-10T22:34:02.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T22:34:02.380 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T22:34:02.380 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T22:34:02.380 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000129124 s, 4.0 MB/s 2024-09-10T22:34:02.381 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T22:34:02.440 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 22:33:45.852955832 +0000 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 22:31:52.454417928 +0000 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 22:31:52.454417928 +0000 2024-09-10T22:34:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 22:31:52.454417928 +0000 2024-09-10T22:34:02.499 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T22:34:02.564 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T22:34:02.564 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T22:34:02.564 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.556e-05 s, 5.4 MB/s 2024-09-10T22:34:02.565 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T22:34:02.623 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-09-10T22:34:02.680 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T22:34:02.680 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:02.680 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T22:34:02.680 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:02.681 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:02.681 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 22:33:45.852955832 +0000 2024-09-10T22:34:02.681 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 22:31:52.762412732 +0000 2024-09-10T22:34:02.681 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 22:31:52.762412732 +0000 2024-09-10T22:34:02.681 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 22:31:52.762412732 +0000 2024-09-10T22:34:02.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T22:34:02.745 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T22:34:02.745 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T22:34:02.745 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.8343e-05 s, 5.8 MB/s 2024-09-10T22:34:02.746 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T22:34:02.803 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-09-10T22:34:02.859 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T22:34:02.859 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T22:34:02.859 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T22:34:02.859 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:34:02.859 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T22:34:02.859 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-10 22:33:45.853955815 +0000 2024-09-10T22:34:02.860 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-10 22:31:53.065407620 +0000 2024-09-10T22:34:02.860 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-10 22:31:53.065407620 +0000 2024-09-10T22:34:02.860 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-10 22:31:53.065407620 +0000 2024-09-10T22:34:02.860 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T22:34:02.922 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-10T22:34:02.922 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-10T22:34:02.922 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.00010542 s, 4.9 MB/s 2024-09-10T22:34:02.924 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T22:34:02.984 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:02.985 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T22:34:02.985 INFO:tasks.ceph:Generating config... 2024-09-10T22:34:02.986 INFO:tasks.ceph:[client] pid file = 2024-09-10T22:34:02.986 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T22:34:02.986 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T22:34:02.986 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T22:34:02.987 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T22:34:02.987 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T22:34:02.987 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T22:34:02.987 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T22:34:02.987 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T22:34:02.987 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T22:34:02.987 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T22:34:02.987 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T22:34:02.987 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T22:34:02.987 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T22:34:02.988 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T22:34:02.988 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T22:34:02.989 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T22:34:03.043 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T22:34:03.046 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T22:34:03.094 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T22:34:03.122 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.66'), ('mon.b', '172.21.15.94'), ('mon.c', '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]')] 2024-09-10T22:34:03.123 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-10T22:34:03.124 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:03.124 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T22:34:03.188 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.66 --add b 172.21.15.94 --addv c '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:03.317 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:03.317 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: generated fsid 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:34:03.317 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = quincy 2024-09-10T22:34:03.317 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 0 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:fsid 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:last_changed 2024-09-10T22:34:03.316789+0000 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:created 2024-09-10T22:34:03.316789+0000 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:min_mon_release 17 (quincy) 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:election_strategy: 1 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:0: [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon.a 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:1: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:2: [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] mon.c 2024-09-10T22:34:03.318 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T22:34:03.319 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T22:34:03.335 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7ef137cd-a1ce-4819-b27d-b71268612f85... 2024-09-10T22:34:03.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:34:03.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:34:03.420 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.421 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:34:03.422 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-09-10T22:34:03.423 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on skipped message = False 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-09-10T22:34:03.424 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.425 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-09-10T22:34:03.426 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 30 2024-09-10T22:34:03.427 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.428 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-09-10T22:34:03.429 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T22:34:03.430 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T22:34:03.432 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-10T22:34:03.432 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-10T22:34:03.432 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:34:03.432 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-10T22:34:03.433 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:34:03.434 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on skipped message = False 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:34:03.435 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-10T22:34:03.436 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-10T22:34:03.437 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:34:03.438 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 30 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-10T22:34:03.439 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:34:03.440 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-10T22:34:03.441 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T22:34:03.442 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T22:34:03.443 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T22:34:03.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T22:34:03.529 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T22:34:03.530 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:03.530 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T22:34:03.552 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:03.552 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T22:34:03.608 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:34:03.608 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:03.608 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T22:34:03.608 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T22:34:03.686 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:03.686 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:03.745 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:34:03.746 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:03.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T22:34:03.746 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T22:34:03.781 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:03.781 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:03.836 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T22:34:03.837 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T22:34:03.837 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T22:34:03.893 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T22:34:03.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T22:34:03.958 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T22:34:03.960 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T22:34:03.960 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T22:34:04.013 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T22:34:04.016 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T22:34:04.041 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T22:34:04.135 INFO:teuthology.orchestra.run.smithi066.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T22:34:04.138 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T22:34:04.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T22:34:04.214 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T22:34:04.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T22:34:04.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T22:34:04.334 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T22:34:04.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T22:34:04.362 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T22:34:04.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T22:34:04.406 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T22:34:04.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T22:34:04.502 INFO:teuthology.orchestra.run.smithi066.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T22:34:04.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T22:34:04.557 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T22:34:04.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T22:34:04.651 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T22:34:04.664 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T22:34:04.664 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi066.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T22:34:04.664 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T22:34:04.689 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:04.690 INFO:tasks.ceph:role: osd.0 2024-09-10T22:34:04.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:34:04.690 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:04.763 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:05.267 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T22:34:05.269 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T22:34:05.269 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T22:34:05.413 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T22:34:05.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T22:34:05.513 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:05.513 INFO:tasks.ceph:role: osd.1 2024-09-10T22:34:05.513 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:34:05.513 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T22:34:05.587 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:05.587 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:05.587 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:05.588 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:05.588 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:05.588 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:05.588 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:05.588 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:05.588 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:05.588 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:06.031 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T22:34:06.032 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T22:34:06.033 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T22:34:06.124 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T22:34:06.155 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T22:34:06.221 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:06.221 INFO:tasks.ceph:role: osd.2 2024-09-10T22:34:06.221 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:34:06.221 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T22:34:06.294 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:06.294 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:06.294 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:06.294 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:06.294 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:06.295 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:06.295 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:06.295 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:06.295 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:06.295 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:06.795 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T22:34:06.797 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T22:34:06.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T22:34:06.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T22:34:06.950 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T22:34:07.016 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:07.017 INFO:tasks.ceph:role: osd.3 2024-09-10T22:34:07.017 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:34:07.017 DEBUG:teuthology.orchestra.run.smithi066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T22:34:07.090 INFO:teuthology.orchestra.run.smithi066.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:07.090 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:07.090 INFO:teuthology.orchestra.run.smithi066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:07.090 INFO:teuthology.orchestra.run.smithi066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:07.090 INFO:teuthology.orchestra.run.smithi066.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:07.091 INFO:teuthology.orchestra.run.smithi066.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:07.091 INFO:teuthology.orchestra.run.smithi066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:07.091 INFO:teuthology.orchestra.run.smithi066.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:07.091 INFO:teuthology.orchestra.run.smithi066.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:07.091 INFO:teuthology.orchestra.run.smithi066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stdout:Discarding blocks...Done. 2024-09-10T22:34:07.602 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi066.front.sepia.ceph.com -o noatime 2024-09-10T22:34:07.602 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T22:34:07.741 DEBUG:teuthology.orchestra.run.smithi066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T22:34:07.775 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:07.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:07.871+0000 7fef39558640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T22:34:07.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:07.871+0000 7fef39558640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T22:34:07.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:07.871+0000 7fef39558640 -1 bdev(0x55e8466c7800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:07.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:07.871+0000 7fef39558640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T22:34:10.787 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T22:34:10.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:10.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:10.911+0000 7f778d00d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T22:34:10.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:10.911+0000 7f778d00d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T22:34:10.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:10.911+0000 7f778d00d640 -1 bdev(0x55877f435800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:10.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:10.911+0000 7f778d00d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T22:34:14.328 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T22:34:14.359 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:14.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:14.450+0000 7f5ab3811640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T22:34:14.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:14.450+0000 7f5ab3811640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T22:34:14.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:14.450+0000 7f5ab3811640 -1 bdev(0x55ed34733800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:14.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:14.450+0000 7f5ab3811640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T22:34:17.621 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T22:34:17.651 DEBUG:teuthology.orchestra.run.smithi066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:17.743+0000 7f9e1880b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T22:34:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:17.744+0000 7f9e1880b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T22:34:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:17.744+0000 7f9e1880b640 -1 bdev(0x55b923cbb800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:17.744+0000 7f9e1880b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T22:34:20.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T22:34:20.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T22:34:20.721 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:20.721 INFO:tasks.ceph:role: osd.4 2024-09-10T22:34:20.722 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:34:20.722 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T22:34:20.790 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:20.790 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:20.790 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:20.790 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:20.790 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:20.790 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:20.791 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:20.791 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:20.791 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:20.791 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:20.829 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T22:34:20.831 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T22:34:20.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T22:34:20.903 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T22:34:20.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T22:34:21.040 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:21.040 INFO:tasks.ceph:role: osd.5 2024-09-10T22:34:21.040 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:34:21.040 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T22:34:21.104 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:21.105 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:21.146 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T22:34:21.148 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T22:34:21.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T22:34:21.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T22:34:21.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T22:34:21.357 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:21.357 INFO:tasks.ceph:role: osd.6 2024-09-10T22:34:21.357 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:34:21.357 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:21.423 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:21.462 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T22:34:21.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T22:34:21.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T22:34:21.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T22:34:21.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T22:34:21.666 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T22:34:21.666 INFO:tasks.ceph:role: osd.7 2024-09-10T22:34:21.666 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:34:21.666 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T22:34:21.731 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T22:34:21.769 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-10T22:34:21.771 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-10T22:34:21.771 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T22:34:21.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T22:34:21.909 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:22.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:22.000+0000 7fdc5f422640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T22:34:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:22.000+0000 7fdc5f422640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T22:34:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:22.000+0000 7fdc5f422640 -1 bdev(0x55c770b4f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:22.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:22.000+0000 7fdc5f422640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T22:34:24.913 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T22:34:24.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:25.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:25.033+0000 7efd1e422640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T22:34:25.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:25.033+0000 7efd1e422640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T22:34:25.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:25.033+0000 7efd1e422640 -1 bdev(0x5609918a1800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:25.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:25.033+0000 7efd1e422640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T22:34:28.434 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T22:34:28.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:28.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:28.552+0000 7fe1f5e11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T22:34:28.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:28.552+0000 7fe1f5e11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T22:34:28.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:28.552+0000 7fe1f5e11640 -1 bdev(0x556390705800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:28.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:28.552+0000 7fe1f5e11640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T22:34:31.377 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:34:31.380 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:34:31.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T22:34:31.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:32.085+0000 7f54b3d32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T22:34:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:32.085+0000 7f54b3d32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T22:34:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:32.086+0000 7f54b3d32640 -1 bdev(0x55aa00e4b800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T22:34:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:32.086+0000 7f54b3d32640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T22:34:35.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T22:34:35.269 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T22:34:35.269 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:35.269 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T22:34:35.301 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:35.301 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T22:34:35.372 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:35.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T22:34:35.444 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:35.445 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T22:34:35.514 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:35.514 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T22:34:35.582 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:35.582 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T22:34:35.649 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:35.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T22:34:35.717 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:35.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T22:34:35.747 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:35.747 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T22:34:35.816 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:35.816 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T22:34:35.884 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:35.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T22:34:35.950 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:35.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T22:34:36.017 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:36.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T22:34:36.082 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:36.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T22:34:36.148 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:36.148 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T22:34:36.166 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:36.166 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T22:34:36.224 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:36.225 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T22:34:36.240 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:36.240 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T22:34:36.296 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T22:34:36.296 DEBUG:teuthology.orchestra.run.smithi066:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T22:34:36.299 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T22:34:36.339 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.y] 2024-09-10T22:34:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbyeBmchE7NRAAGJXMPuxRS+9kbKAHx1KX4A== 2024-09-10T22:34:36.339 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T22:34:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmOynNABAAKH0qcVEjgabxdgbcMzFenw== 2024-09-10T22:34:36.339 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T22:34:36.339 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmJyYSCBAAHOWAMRZJfptyuhaBDmMdHg== 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.0] 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBfyeBmCe8JNBAANHCDeqvquGRJDSdZEF/nKg== 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.1] 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBiyeBmAHFpNhAAZ/NEsvKX+rtDoru1DhdjMQ== 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.2] 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBmyeBmVeTpGhAAtIciTFU74AgMICAgqvD0/g== 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.3] 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBpyeBm/01eLBAAeiodosiLdSg26EyvnT+OeA== 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr.x] 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBbyeBm80AWORAA002Ap5bZLKASMXqSSM15dw== 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBm92zKDBAAU/37cpMQbv3hwxpOOF7wqw== 2024-09-10T22:34:36.340 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBm9InwExAAOmJR0PSYaLknUYrjqdx8aQ== 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.4] 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBuyeBmyrwOABAAx2o/hEASH4yqt4ZmQA68Wg== 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.5] 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBxyeBmIZEGAhAAd8+PqJWUZGZyB7BHmz60xw== 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.6] 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB0yeBmvBX4IBAAH7DccJtEh0K9AYIrb/1GjA== 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:[osd.7] 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB4yeBmGD4nBRAAWQ51L9Bx7x6P0dwTRRFe8Q== 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmyY04GBAAcxW03/29zpbpxXzE+JrsFA== 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout:[client.2] 2024-09-10T22:34:36.341 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmasn2HRAAv24FRmz9CPJsGJ8DA1hOIQ== 2024-09-10T22:34:36.342 INFO:teuthology.orchestra.run.smithi066.stdout:[client.1] 2024-09-10T22:34:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmtjE1IRAAO/ItVtJZ2xGcQj/LcTecMQ== 2024-09-10T22:34:36.342 INFO:teuthology.orchestra.run.smithi066.stdout:[client.3] 2024-09-10T22:34:36.342 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmz6LSJhAASEiDnC8gX9owU8E46MLj/w== 2024-09-10T22:34:36.358 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBbyeBmchE7NRAAGJXMPuxRS+9kbKAHx1KX4A== 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBmOynNABAAKH0qcVEjgabxdgbcMzFenw== 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBmJyYSCBAAHOWAMRZJfptyuhaBDmMdHg== 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBfyeBmCe8JNBAANHCDeqvquGRJDSdZEF/nKg== 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBiyeBmAHFpNhAAZ/NEsvKX+rtDoru1DhdjMQ== 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBmyeBmVeTpGhAAtIciTFU74AgMICAgqvD0/g== 2024-09-10T22:34:36.359 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBpyeBm/01eLBAAeiodosiLdSg26EyvnT+OeA== 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBbyeBm80AWORAA002Ap5bZLKASMXqSSM15dw== 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBm92zKDBAAU/37cpMQbv3hwxpOOF7wqw== 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBm9InwExAAOmJR0PSYaLknUYrjqdx8aQ== 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBuyeBmyrwOABAAx2o/hEASH4yqt4ZmQA68Wg== 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBxyeBmIZEGAhAAd8+PqJWUZGZyB7BHmz60xw== 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQB0yeBmvBX4IBAAH7DccJtEh0K9AYIrb/1GjA== 2024-09-10T22:34:36.360 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQB4yeBmGD4nBRAAWQ51L9Bx7x6P0dwTRRFe8Q== 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBmyY04GBAAcxW03/29zpbpxXzE+JrsFA== 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout:[client.2] 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBmasn2HRAAv24FRmz9CPJsGJ8DA1hOIQ== 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBmtjE1IRAAO/ItVtJZ2xGcQj/LcTecMQ== 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout:[client.3] 2024-09-10T22:34:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQBcyeBmz6LSJhAASEiDnC8gX9owU8E46MLj/w== 2024-09-10T22:34:36.362 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T22:34:36.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T22:34:36.453 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.455 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.510 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.568 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.570 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.621 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.730 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.784 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T22:34:36.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T22:34:36.837 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.894 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T22:34:36.950 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:36.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:37.004 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:37.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:37.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:37.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:37.111 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:37.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T22:34:37.165 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.221 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.279 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.281 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.335 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T22:34:37.392 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T22:34:37.392 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T22:34:37.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T22:34:37.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T22:34:37.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T22:34:37.568 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T22:34:37.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T22:34:37.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T22:34:37.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T22:34:37.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T22:34:37.872 DEBUG:teuthology.orchestra.run.smithi066:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:37.874 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T22:34:37.927 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T22:34:37.928 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T22:34:37.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T22:34:37.931 INFO:tasks.ceph.mon.a:Started 2024-09-10T22:34:37.931 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T22:34:37.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T22:34:37.970 INFO:tasks.ceph.mon.b:Started 2024-09-10T22:34:37.971 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T22:34:37.971 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T22:34:37.973 INFO:tasks.ceph.mon.c:Started 2024-09-10T22:34:37.973 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T22:34:37.973 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T22:34:37.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T22:34:37.976 INFO:tasks.ceph.mgr.y:Started 2024-09-10T22:34:37.976 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T22:34:37.977 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T22:34:37.978 INFO:tasks.ceph.mgr.x:Started 2024-09-10T22:34:37.979 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T22:34:37.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T22:34:37.981 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T22:34:37.981 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T22:34:38.025 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T22:34:38.025 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T22:34:38.068 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T22:34:38.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T22:34:38.115 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T22:34:38.115 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T22:34:38.158 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T22:34:38.158 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T22:34:38.201 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T22:34:38.201 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T22:34:38.245 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T22:34:38.245 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T22:34:38.288 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T22:34:38.289 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T22:34:38.332 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T22:34:38.332 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T22:34:38.382 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T22:34:38.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T22:34:38.428 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T22:34:38.429 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T22:34:38.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.425+0000 7f514d7d3640 1 -- 172.21.15.66:0/618963394 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51481e3f70 msgr2=0x7f5148070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.425+0000 7f514d7d3640 1 --2- 172.21.15.66:0/618963394 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51481e3f70 0x7f5148070d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f513c001eb0 tx=0x7f5148073cc0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.425+0000 7f514d7d3640 1 -- 172.21.15.66:0/618963394 shutdown_connections 2024-09-10T22:34:38.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.425+0000 7f514d7d3640 1 --2- 172.21.15.66:0/618963394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f514813a3f0 0x7f5148071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.425+0000 7f514d7d3640 1 --2- 172.21.15.66:0/618963394 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f51481e3f70 0x7f5148070d20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.425+0000 7f514d7d3640 1 --2- 172.21.15.66:0/618963394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5130004800 0x7f51481515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.425+0000 7f514d7d3640 1 -- 172.21.15.66:0/618963394 >> 172.21.15.66:0/618963394 conn(0x7f51480b27c0 msgr2=0x7f5148158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.429+0000 7f514d7d3640 1 -- 172.21.15.66:0/618963394 shutdown_connections 2024-09-10T22:34:38.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.429+0000 7f514d7d3640 1 -- 172.21.15.66:0/618963394 wait complete. 2024-09-10T22:34:38.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.430+0000 7f5c7d14b640 1 -- 172.21.15.66:0/4174081863 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f5c50005310 con 0x7f5c78073bb0 2024-09-10T22:34:38.438 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T22:34:38.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T22:34:38.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.449+0000 7f5c5f7fe640 1 -- 172.21.15.66:0/4174081863 <== mon.1 v2:172.21.15.94:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f5c6800f940 con 0x7f5c78073bb0 2024-09-10T22:34:38.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.449+0000 7f5c5f7fe640 1 -- 172.21.15.66:0/4174081863 <== mon.1 v2:172.21.15.94:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f5c68014030 con 0x7f5c78073bb0 2024-09-10T22:34:38.454 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T22:34:38.454 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T22:34:38.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.449+0000 7f5fb3fff640 1 -- 172.21.15.66:0/3822071003 <== mon.0 v2:172.21.15.66:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f5fc00243f0 con 0x7f5fcc0d0400 2024-09-10T22:34:38.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.451+0000 7fe1177fe640 1 -- 172.21.15.66:0/3498520649 <== mon.0 v2:172.21.15.66:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fe128047460 con 0x7fe13016f170 2024-09-10T22:34:38.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.453+0000 7fc45a808640 1 -- 172.21.15.66:0/1254991998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4541506e0 msgr2=0x7fc454150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.453+0000 7fc45a808640 1 --2- 172.21.15.66:0/1254991998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4541506e0 0x7fc454150b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009830 tx=0x7fc43c02a480 comp rx=0 tx=0).stop 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7f5c7d14b640 1 -- 172.21.15.66:0/4174081863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c78073bb0 msgr2=0x7f5c7814b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7f5c7d14b640 1 --2- 172.21.15.66:0/4174081863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c78073bb0 0x7f5c7814b360 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c740 tx=0x7f5c6800cc00 comp rx=0 tx=0).stop 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7f5c7d14b640 1 -- 172.21.15.66:0/4174081863 shutdown_connections 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7f5c7d14b640 1 --2- 172.21.15.66:0/4174081863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c78151f80 0x7f5c781503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7f5c7d14b640 1 --2- 172.21.15.66:0/4174081863 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c780b4150 0x7f5c7814feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7f5c7d14b640 1 --2- 172.21.15.66:0/4174081863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c78073bb0 0x7f5c7814b360 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7f5c7d14b640 1 -- 172.21.15.66:0/4174081863 >> 172.21.15.66:0/4174081863 conn(0x7f5c780b27c0 msgr2=0x7f5c7814cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7fc45a808640 1 -- 172.21.15.66:0/1254991998 shutdown_connections 2024-09-10T22:34:38.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7fc45a808640 1 --2- 172.21.15.66:0/1254991998 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4541506e0 0x7fc454150b10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7fc45a808640 1 --2- 172.21.15.66:0/1254991998 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cd960 0x7fc45406e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7fc45a808640 1 -- 172.21.15.66:0/1254991998 >> 172.21.15.66:0/1254991998 conn(0x7fc4540b16d0 msgr2=0x7fc4540b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7fc45a808640 1 -- 172.21.15.66:0/1254991998 shutdown_connections 2024-09-10T22:34:38.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.457+0000 7fc45a808640 1 -- 172.21.15.66:0/1254991998 wait complete. 2024-09-10T22:34:38.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.459+0000 7fc45a808640 1 Processor -- start 2024-09-10T22:34:38.464 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T22:34:38.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T22:34:38.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.461+0000 7f5c7d14b640 1 -- 172.21.15.66:0/4174081863 shutdown_connections 2024-09-10T22:34:38.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.461+0000 7f5c7d14b640 1 -- 172.21.15.66:0/4174081863 wait complete. 2024-09-10T22:34:38.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.461+0000 7fc45a808640 1 -- start start 2024-09-10T22:34:38.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.462+0000 7fc45a808640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4540cd960 0x7fc4540aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.462+0000 7fc45a808640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cdd40 0x7fc4540c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.462+0000 7fc45a808640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4541506e0 0x7fc4540b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.462+0000 7fc45a808640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc454151420 con 0x7fc4540cd960 2024-09-10T22:34:38.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.462+0000 7fc45a808640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541512a0 con 0x7fc4541506e0 2024-09-10T22:34:38.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.462+0000 7fc45a808640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc454070430 con 0x7fc4540cdd40 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cdd40 0x7fc4540c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cdd40 0x7fc4540c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:40752/0 (socket says 172.21.15.66:40752) 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 -- 172.21.15.66:0/2674418539 learned_addr learned my addr 172.21.15.66:0/2674418539 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc458da5640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4541506e0 0x7fc4540b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc453fff640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4540cd960 0x7fc4540aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 -- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4541506e0 msgr2=0x7fc4540b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4541506e0 0x7fc4540b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 -- 172.21.15.66:0/2674418539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4540cd960 msgr2=0x7fc4540aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4540cd960 0x7fc4540aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.463+0000 7fc4537fe640 1 -- 172.21.15.66:0/2674418539 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4540c5ac0 con 0x7fc4540cdd40 2024-09-10T22:34:38.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.464+0000 7fc4537fe640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cdd40 0x7fc4540c4fe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00cbd0 tx=0x7fc44c00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.470+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c016020 con 0x7fc4540cdd40 2024-09-10T22:34:38.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.470+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4540c5da0 con 0x7fc4540cdd40 2024-09-10T22:34:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.470+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4540c6360 con 0x7fc4540cdd40 2024-09-10T22:34:38.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.471+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4540731e0 con 0x7fc4540cdd40 2024-09-10T22:34:38.476 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T22:34:38.477 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T22:34:38.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.476+0000 7f5fd1b89640 1 -- 172.21.15.66:0/3822071003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f5fa4005310 con 0x7f5fcc0d0400 2024-09-10T22:34:38.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.470+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fc44c00ae10 con 0x7fc4540cdd40 2024-09-10T22:34:38.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.476+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c012680 con 0x7fc4540cdd40 2024-09-10T22:34:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.478+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fc44c012820 con 0x7fc4540cdd40 2024-09-10T22:34:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.478+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc44c005820 con 0x7fc4540cdd40 2024-09-10T22:34:38.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.478+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fc4540731e0 con 0x7fc4540cdd40 2024-09-10T22:34:38.488 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T22:34:38.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T22:34:38.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.484+0000 7f5fb3fff640 1 -- 172.21.15.66:0/3822071003 <== mon.0 v2:172.21.15.66:3300/0 11 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f5fc001b9b0 con 0x7f5fcc0d0400 2024-09-10T22:34:38.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.484+0000 7f5fb3fff640 1 -- 172.21.15.66:0/3822071003 <== mon.0 v2:172.21.15.66:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fc001bcd0 con 0x7f5fcc0d0400 2024-09-10T22:34:38.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.488+0000 7f5fd1b89640 1 -- 172.21.15.66:0/3822071003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fcc0d0400 msgr2=0x7f5fcc1439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.488+0000 7f5fd1b89640 1 --2- 172.21.15.66:0/3822071003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fcc0d0400 0x7f5fcc1439c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0010b30 tx=0x7f5fc000e350 comp rx=0 tx=0).stop 2024-09-10T22:34:38.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.489+0000 7f5fd1b89640 1 -- 172.21.15.66:0/3822071003 shutdown_connections 2024-09-10T22:34:38.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.489+0000 7f5fd1b89640 1 --2- 172.21.15.66:0/3822071003 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fcc14b5c0 0x7f5fcc143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.489+0000 7f5fd1b89640 1 --2- 172.21.15.66:0/3822071003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fcc0d0400 0x7f5fcc1439c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.489+0000 7f5fd1b89640 1 --2- 172.21.15.66:0/3822071003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fcc0cd4b0 0x7f5fcc08b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.489+0000 7f5fd1b89640 1 -- 172.21.15.66:0/3822071003 >> 172.21.15.66:0/3822071003 conn(0x7f5fcc06dc30 msgr2=0x7f5fcc15d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.489+0000 7f5fd1b89640 1 -- 172.21.15.66:0/3822071003 shutdown_connections 2024-09-10T22:34:38.492 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T22:34:38.492 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T22:34:38.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.490+0000 7fe1177fe640 1 -- 172.21.15.66:0/3498520649 <== mon.0 v2:172.21.15.66:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe128051430 con 0x7fe13016f170 2024-09-10T22:34:38.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.492+0000 7f5fd1b89640 1 -- 172.21.15.66:0/3822071003 wait complete. 2024-09-10T22:34:38.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.497+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc44c012b40 con 0x7fc4540cdd40 2024-09-10T22:34:38.506 DEBUG:tasks.ceph:set 18 configs 2024-09-10T22:34:38.506 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.543 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.544 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.544 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.544 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.544 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.555+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/1145351917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c80004800 msgr2=0x7f5c80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.555+0000 7f5c9cb8a640 1 --2- 172.21.15.66:0/1145351917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c80004800 0x7f5c80004c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c680 tx=0x7f5c80005830 comp rx=0 tx=0).stop 2024-09-10T22:34:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.558+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/1145351917 shutdown_connections 2024-09-10T22:34:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.558+0000 7f5c9cb8a640 1 --2- 172.21.15.66:0/1145351917 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c80004800 0x7f5c80004c80 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c680 tx=0x7f5c80005830 comp rx=0 tx=0).stop 2024-09-10T22:34:38.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.559+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/1145351917 >> 172.21.15.66:0/1145351917 conn(0x7f5c9806d230 msgr2=0x7f5c980b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.567+0000 7f568c7ca640 1 -- 172.21.15.66:0/1391765549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f567c0e6dc0 msgr2=0x7f567c036850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.567+0000 7f568c7ca640 1 --2- 172.21.15.66:0/1391765549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f567c0e6dc0 0x7f567c036850 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f567c035e30 tx=0x7f56840b6bd0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.567+0000 7f568c7ca640 1 -- 172.21.15.66:0/1391765549 shutdown_connections 2024-09-10T22:34:38.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.567+0000 7f568c7ca640 1 --2- 172.21.15.66:0/1391765549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f567c0e6dc0 0x7f567c036850 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.567+0000 7f568c7ca640 1 -- 172.21.15.66:0/1391765549 >> 172.21.15.66:0/1391765549 conn(0x7f567c01d7a0 msgr2=0x7f567c054590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.569+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/1145351917 shutdown_connections 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.571+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/1145351917 wait complete. 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.573+0000 7f5c9cb8a640 1 Processor -- start 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.573+0000 7f5c9cb8a640 1 -- start start 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.574+0000 7f5c9cb8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c9815add0 0x7f5c980af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.574+0000 7f5c9cb8a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980b0b10 0x7f5c981f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.574+0000 7f5c9cb8a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980cc340 0x7f5c980cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980b0b10 0x7f5c981f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980b0b10 0x7f5c981f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:40768/0 (socket says 172.21.15.66:40768) 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c9815add0 0x7f5c980af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c977fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c9815add0 0x7f5c980af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:48380/0 (socket says 172.21.15.66:48380) 2024-09-10T22:34:38.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f568c7ca640 1 -- 172.21.15.66:0/1391765549 shutdown_connections 2024-09-10T22:34:38.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980cc340 0x7f5c980cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980cc340 0x7f5c980cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57124/0 (socket says 172.21.15.66:57124) 2024-09-10T22:34:38.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c9cb8a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c980b3990 con 0x7f5c980cc340 2024-09-10T22:34:38.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c9cb8a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9815f0b0 con 0x7f5c9815add0 2024-09-10T22:34:38.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.575+0000 7f5c9cb8a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5c980b3b10 con 0x7f5c980b0b10 2024-09-10T22:34:38.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.576+0000 7f5c96ffd640 1 -- 172.21.15.66:0/3780158152 learned_addr learned my addr 172.21.15.66:0/3780158152 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.576+0000 7f5c96ffd640 1 -- 172.21.15.66:0/3780158152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c9815add0 msgr2=0x7f5c980af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.576+0000 7f5c96ffd640 1 --2- 172.21.15.66:0/3780158152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c9815add0 0x7f5c980af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.576+0000 7f5c96ffd640 1 -- 172.21.15.66:0/3780158152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980cc340 msgr2=0x7f5c980cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.576+0000 7f5c96ffd640 1 --2- 172.21.15.66:0/3780158152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980cc340 0x7f5c980cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.576+0000 7f5c96ffd640 1 -- 172.21.15.66:0/3780158152 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c980afa90 con 0x7f5c980b0b10 2024-09-10T22:34:38.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.578+0000 7f568c7ca640 1 -- 172.21.15.66:0/1391765549 wait complete. 2024-09-10T22:34:38.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.579+0000 7f5c96ffd640 1 --2- 172.21.15.66:0/3780158152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980b0b10 0x7f5c981f5050 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c680 tx=0x7f5c90001aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.579+0000 7f568c7ca640 1 Processor -- start 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.579+0000 7f568c7ca640 1 -- start start 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568c7ca640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f567c04eb50 0x7f567c04a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568c7ca640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f567c0e69e0 0x7f567c048030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568c7ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c0d25e0 0x7f567c048570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568c7ca640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0e4920 con 0x7f567c0e69e0 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568c7ca640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f567c055310 con 0x7f567c0d25e0 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568c7ca640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f567c0e4aa0 con 0x7f567c04eb50 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568ad67640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c0d25e0 0x7f567c048570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568ad67640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c0d25e0 0x7f567c048570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:48384/0 (socket says 172.21.15.66:48384) 2024-09-10T22:34:38.583 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.581+0000 7f568ad67640 1 -- 172.21.15.66:0/3351967847 learned_addr learned my addr 172.21.15.66:0/3351967847 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.585+0000 7f5c7ffff640 1 -- 172.21.15.66:0/3780158152 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90042070 con 0x7f5c980b0b10 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.585+0000 7f5c7ffff640 1 -- 172.21.15.66:0/3780158152 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c9002bcd0 con 0x7f5c980b0b10 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.585+0000 7f5c7ffff640 1 -- 172.21.15.66:0/3780158152 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9000c2a0 con 0x7f5c980b0b10 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f568ad67640 1 -- 172.21.15.66:0/3351967847 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f567c04eb50 msgr2=0x7f567c04a990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f568ad67640 1 --2- 172.21.15.66:0/3351967847 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f567c04eb50 0x7f567c04a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f568ad67640 1 -- 172.21.15.66:0/3351967847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f567c0e69e0 msgr2=0x7f567c048030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f568ad67640 1 --2- 172.21.15.66:0/3351967847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f567c0e69e0 0x7f567c048030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f568ad67640 1 -- 172.21.15.66:0/3351967847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c0e3a20 con 0x7f567c0d25e0 2024-09-10T22:34:38.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f568ad67640 1 --2- 172.21.15.66:0/3351967847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c0d25e0 0x7f567c048570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f56840668e0 tx=0x7f56840b7ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f567affd640 1 -- 172.21.15.66:0/3351967847 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684088070 con 0x7f567c0d25e0 2024-09-10T22:34:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f567affd640 1 -- 172.21.15.66:0/3351967847 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56840bb050 con 0x7f567c0d25e0 2024-09-10T22:34:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f567affd640 1 -- 172.21.15.66:0/3351967847 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568407f070 con 0x7f567c0d25e0 2024-09-10T22:34:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.582+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c0e3d00 con 0x7f567c0d25e0 2024-09-10T22:34:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.583+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c052290 con 0x7f567c0d25e0 2024-09-10T22:34:38.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.583+0000 7f567affd640 1 -- 172.21.15.66:0/3351967847 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f56840bf410 con 0x7f567c0d25e0 2024-09-10T22:34:38.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.588+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/3780158152 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c980afd70 con 0x7f5c980b0b10 2024-09-10T22:34:38.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.588+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/3780158152 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c980b0330 con 0x7f5c980b0b10 2024-09-10T22:34:38.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.590+0000 7f5c7dffb640 1 -- 172.21.15.66:0/3780158152 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c78005290 con 0x7f5c980b0b10 2024-09-10T22:34:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.591+0000 7f567affd640 1 -- 172.21.15.66:0/3351967847 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5684063590 con 0x7f567c0d25e0 2024-09-10T22:34:38.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.591+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c005290 con 0x7f567c0d25e0 2024-09-10T22:34:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.595+0000 7f5c7ffff640 1 -- 172.21.15.66:0/3780158152 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5c900062d0 con 0x7f5c980b0b10 2024-09-10T22:34:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.595+0000 7f5c7ffff640 1 -- 172.21.15.66:0/3780158152 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5c9003dbf0 con 0x7f5c980b0b10 2024-09-10T22:34:38.596 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.595+0000 7f5c7ffff640 1 -- 172.21.15.66:0/3780158152 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5c9004b020 con 0x7f5c980b0b10 2024-09-10T22:34:38.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.608+0000 7f567affd640 1 -- 172.21.15.66:0/3351967847 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f56840be820 con 0x7f567c0d25e0 2024-09-10T22:34:38.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.671+0000 7fe1350ed640 1 -- 172.21.15.66:0/3498520649 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fe110004300 con 0x7fe13016f170 2024-09-10T22:34:38.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.678+0000 7fe1177fe640 1 -- 172.21.15.66:0/3498520649 <== mon.0 v2:172.21.15.66:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7fe1280517d0 con 0x7fe13016f170 2024-09-10T22:34:38.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.694+0000 7f42834dd640 1 -- 172.21.15.66:0/4132843545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c08ba70 msgr2=0x7f427c070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.694+0000 7f42834dd640 1 --2- 172.21.15.66:0/4132843545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c08ba70 0x7f427c070890 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4278009880 tx=0x7f427802a4d0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.694+0000 7f42834dd640 1 -- 172.21.15.66:0/4132843545 shutdown_connections 2024-09-10T22:34:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.694+0000 7f42834dd640 1 --2- 172.21.15.66:0/4132843545 >> v2:172.21.15.94:3300/0 conn(0x7f427c0b41f0 0x7f427c167c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.694+0000 7f42834dd640 1 --2- 172.21.15.66:0/4132843545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c08ba70 0x7f427c070890 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.694+0000 7f42834dd640 1 -- 172.21.15.66:0/4132843545 >> 172.21.15.66:0/4132843545 conn(0x7f427c0cfaa0 msgr2=0x7f427c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.695+0000 7fe1350ed640 1 -- 172.21.15.66:0/3498520649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13016f170 msgr2=0x7fe1301f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.695+0000 7fe1350ed640 1 --2- 172.21.15.66:0/3498520649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13016f170 0x7fe1301f5bd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe12802b9f0 tx=0x7fe128004a40 comp rx=0 tx=0).stop 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.695+0000 7f42834dd640 1 -- 172.21.15.66:0/4132843545 shutdown_connections 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.695+0000 7f42834dd640 1 -- 172.21.15.66:0/4132843545 wait complete. 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.697+0000 7f42834dd640 1 Processor -- start 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.697+0000 7f42834dd640 1 -- start start 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.698+0000 7f42834dd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f427c0b41f0 0x7f427c08acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.698+0000 7f42834dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c16b9f0 0x7f427c073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.698+0000 7f42834dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f427c201fc0 0x7f427c0739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.698+0000 7f42834dd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f427c168520 con 0x7f427c16b9f0 2024-09-10T22:34:38.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.698+0000 7f42834dd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1683a0 con 0x7f427c201fc0 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.698+0000 7f42834dd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f427c1686a0 con 0x7f427c0b41f0 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.699+0000 7f4280a78640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c16b9f0 0x7f427c073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.699+0000 7f4280a78640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c16b9f0 0x7f427c073490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57136/0 (socket says 172.21.15.66:57136) 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.699+0000 7f4280a78640 1 -- 172.21.15.66:0/2388326192 learned_addr learned my addr 172.21.15.66:0/2388326192 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.700+0000 7f4280a78640 1 -- 172.21.15.66:0/2388326192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f427c0b41f0 msgr2=0x7f427c08acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.700+0000 7f4280a78640 1 --2- 172.21.15.66:0/2388326192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f427c0b41f0 0x7f427c08acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.700+0000 7f4280a78640 1 -- 172.21.15.66:0/2388326192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f427c201fc0 msgr2=0x7f427c0739d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.700+0000 7f4280a78640 1 --2- 172.21.15.66:0/2388326192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f427c201fc0 0x7f427c0739d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.700+0000 7f4280a78640 1 -- 172.21.15.66:0/2388326192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c2055d0 con 0x7f427c16b9f0 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.700+0000 7f4280a78640 1 --2- 172.21.15.66:0/2388326192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c16b9f0 0x7f427c073490 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f427400cb10 tx=0x7f427400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.701+0000 7f4271ffb640 1 -- 172.21.15.66:0/2388326192 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274017070 con 0x7f427c16b9f0 2024-09-10T22:34:38.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7fe1350ed640 1 -- 172.21.15.66:0/3498520649 shutdown_connections 2024-09-10T22:34:38.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7fe1350ed640 1 --2- 172.21.15.66:0/3498520649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe13016f170 0x7fe1301f5bd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7fe1350ed640 1 --2- 172.21.15.66:0/3498520649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe13016e5d0 0x7fe1301f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7fe1350ed640 1 --2- 172.21.15.66:0/3498520649 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe130073670 0x7fe130168030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7fe1350ed640 1 -- 172.21.15.66:0/3498520649 >> 172.21.15.66:0/3498520649 conn(0x7fe1300b16d0 msgr2=0x7fe13008a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7fe1350ed640 1 -- 172.21.15.66:0/3498520649 shutdown_connections 2024-09-10T22:34:38.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7fe1350ed640 1 -- 172.21.15.66:0/3498520649 wait complete. 2024-09-10T22:34:38.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.701+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c2014e0 con 0x7f427c16b9f0 2024-09-10T22:34:38.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.702+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c201aa0 con 0x7f427c16b9f0 2024-09-10T22:34:38.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.704+0000 7f4271ffb640 1 -- 172.21.15.66:0/2388326192 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4274003c60 con 0x7f427c16b9f0 2024-09-10T22:34:38.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.704+0000 7f4271ffb640 1 -- 172.21.15.66:0/2388326192 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427400aac0 con 0x7f427c16b9f0 2024-09-10T22:34:38.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.705+0000 7f4271ffb640 1 -- 172.21.15.66:0/2388326192 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f427400ace0 con 0x7f427c16b9f0 2024-09-10T22:34:38.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.705+0000 7f4271ffb640 1 -- 172.21.15.66:0/2388326192 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f427401cb60 con 0x7f427c16b9f0 2024-09-10T22:34:38.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.706+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425c005290 con 0x7f427c16b9f0 2024-09-10T22:34:38.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.710+0000 7f4271ffb640 1 -- 172.21.15.66:0/2388326192 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f4274021020 con 0x7f427c16b9f0 2024-09-10T22:34:38.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 -- 172.21.15.66:0/3947662182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11680b41f0 msgr2=0x7f1168070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 --2- 172.21.15.66:0/3947662182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11680b41f0 0x7f1168070890 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1160005fd0 tx=0x7f116002aae0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 -- 172.21.15.66:0/3947662182 shutdown_connections 2024-09-10T22:34:38.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 --2- 172.21.15.66:0/3947662182 >> v2:172.21.15.94:3300/0 conn(0x7f116816b9f0 0x7f1168167c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 --2- 172.21.15.66:0/3947662182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11680b41f0 0x7f1168070890 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 -- 172.21.15.66:0/3947662182 >> 172.21.15.66:0/3947662182 conn(0x7f11680cfaa0 msgr2=0x7f11680cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 -- 172.21.15.66:0/3947662182 shutdown_connections 2024-09-10T22:34:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.718+0000 7f116e117640 1 -- 172.21.15.66:0/3947662182 wait complete. 2024-09-10T22:34:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.720+0000 7f116e117640 1 Processor -- start 2024-09-10T22:34:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.720+0000 7f116e117640 1 -- start start 2024-09-10T22:34:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.721+0000 7f116e117640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f116808ba70 0x7f11680aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.721+0000 7f116e117640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f116816b9f0 0x7f1168200830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.721+0000 7f116e117640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1168200d70 0x7f11680ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.721+0000 7f116e117640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1168168520 con 0x7f1168200d70 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.721+0000 7f116e117640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11681683a0 con 0x7f116816b9f0 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.721+0000 7f116e117640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f11681686a0 con 0x7f116808ba70 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f116808ba70 0x7f11680aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f116808ba70 0x7f11680aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:40792/0 (socket says 172.21.15.66:40792) 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 -- 172.21.15.66:0/3512422871 learned_addr learned my addr 172.21.15.66:0/3512422871 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f1166ffd640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f116816b9f0 0x7f1168200830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 -- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f116816b9f0 msgr2=0x7f1168200830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f116816b9f0 0x7f1168200830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 -- 172.21.15.66:0/3512422871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1168200d70 msgr2=0x7f11680ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f1167fff640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1168200d70 0x7f11680ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1168200d70 0x7f11680ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f11677fe640 1 -- 172.21.15.66:0/3512422871 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116815b300 con 0x7f116808ba70 2024-09-10T22:34:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.722+0000 7f1167fff640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1168200d70 0x7f11680ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:34:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.723+0000 7f11677fe640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f116808ba70 0x7f11680aece0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f115800c480 tx=0x7f115800c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.724+0000 7f0481833640 1 -- 172.21.15.66:0/1480035281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 msgr2=0x7f047c0d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.724+0000 7f114ffff640 1 -- 172.21.15.66:0/3512422871 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11580052d0 con 0x7f116808ba70 2024-09-10T22:34:38.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.724+0000 7f0481833640 1 --2- 172.21.15.66:0/1480035281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 0x7f047c0d4a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f047c0ce740 tx=0x7f047002a420 comp rx=0 tx=0).stop 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.724+0000 7f116e117640 1 -- 172.21.15.66:0/3512422871 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11680af2e0 con 0x7f116808ba70 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.725+0000 7f116e117640 1 -- 172.21.15.66:0/3512422871 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f116815b8a0 con 0x7f116808ba70 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.726+0000 7f114ffff640 1 -- 172.21.15.66:0/3512422871 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1158005470 con 0x7f116808ba70 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.726+0000 7f114ffff640 1 -- 172.21.15.66:0/3512422871 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158012870 con 0x7f116808ba70 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.728+0000 7f116e117640 1 -- 172.21.15.66:0/3512422871 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1148005290 con 0x7f116808ba70 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f114ffff640 1 -- 172.21.15.66:0/3512422871 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1158012ad0 con 0x7f116808ba70 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f114ffff640 1 -- 172.21.15.66:0/3512422871 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f115801b920 con 0x7f116808ba70 2024-09-10T22:34:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f759d852640 1 -- 172.21.15.66:0/27652906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759815e940 msgr2=0x7f75980afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f759d852640 1 --2- 172.21.15.66:0/27652906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759815e940 0x7f75980afc70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f759806ecb0 tx=0x7f758802a4d0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f759d852640 1 -- 172.21.15.66:0/27652906 shutdown_connections 2024-09-10T22:34:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f759d852640 1 --2- 172.21.15.66:0/27652906 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759815e940 0x7f75980afc70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f759d852640 1 --2- 172.21.15.66:0/27652906 >> v2:172.21.15.66:3300/0 conn(0x7f75980cd4f0 0x7f75980cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.729+0000 7f759d852640 1 -- 172.21.15.66:0/27652906 >> 172.21.15.66:0/27652906 conn(0x7f75980b1f00 msgr2=0x7f75980b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.731+0000 7f0481833640 1 -- 172.21.15.66:0/1480035281 shutdown_connections 2024-09-10T22:34:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.731+0000 7f0481833640 1 --2- 172.21.15.66:0/1480035281 >> v2:172.21.15.66:3300/0 conn(0x7f047c0d1f90 0x7f047c0d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.731+0000 7f0481833640 1 --2- 172.21.15.66:0/1480035281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 0x7f047c0d4a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.731+0000 7f0481833640 1 -- 172.21.15.66:0/1480035281 >> 172.21.15.66:0/1480035281 conn(0x7f047c0d3f10 msgr2=0x7f047c0d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.731+0000 7f0481833640 1 -- 172.21.15.66:0/1480035281 shutdown_connections 2024-09-10T22:34:38.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.731+0000 7f0481833640 1 -- 172.21.15.66:0/1480035281 wait complete. 2024-09-10T22:34:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.733+0000 7f0481833640 1 Processor -- start 2024-09-10T22:34:38.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.734+0000 7f0481833640 1 -- start start 2024-09-10T22:34:38.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.735+0000 7f0481833640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 0x7f047c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.735+0000 7f0481833640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f047c0d1f90 0x7f047c153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f047affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 0x7f047c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f047affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 0x7f047c0c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:48436/0 (socket says 172.21.15.66:48436) 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f047a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f047c0d1f90 0x7f047c153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f047a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f047c0d1f90 0x7f047c153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57182/0 (socket says 172.21.15.66:57182) 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f047affd640 1 -- 172.21.15.66:0/3922141828 learned_addr learned my addr 172.21.15.66:0/3922141828 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f0481833640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f047c15e120 0x7f047c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f0481833640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f047c0b70f0 con 0x7f047c0d1f90 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f0481833640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f047c0b6f70 con 0x7f047c0d10d0 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f047b7fe640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f047c15e120 0x7f047c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.736+0000 7f0481833640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f047c0b7270 con 0x7f047c15e120 2024-09-10T22:34:38.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.734+0000 7f114ffff640 1 -- 172.21.15.66:0/3512422871 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1158012df0 con 0x7f116808ba70 2024-09-10T22:34:38.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.739+0000 7f759d852640 1 -- 172.21.15.66:0/27652906 shutdown_connections 2024-09-10T22:34:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.739+0000 7f759d852640 1 -- 172.21.15.66:0/27652906 wait complete. 2024-09-10T22:34:38.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.741+0000 7f759d852640 1 Processor -- start 2024-09-10T22:34:38.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.741+0000 7f759d852640 1 -- start start 2024-09-10T22:34:38.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.742+0000 7f759d852640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75980cd4f0 0x7f75980c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.742+0000 7f759d852640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759815a420 0x7f759806fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.742+0000 7f759d852640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75980b4a50 0x7f759815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.742+0000 7f759d852640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7598068d50 con 0x7f75980b4a50 2024-09-10T22:34:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.742+0000 7f759d852640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7598068bd0 con 0x7f759815a420 2024-09-10T22:34:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.742+0000 7f759d852640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7598068ed0 con 0x7f75980cd4f0 2024-09-10T22:34:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75980cd4f0 0x7f75980c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75980cd4f0 0x7f75980c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:40812/0 (socket says 172.21.15.66:40812) 2024-09-10T22:34:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 -- 172.21.15.66:0/1883981137 learned_addr learned my addr 172.21.15.66:0/1883981137 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 -- 172.21.15.66:0/1883981137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759815a420 msgr2=0x7f759806fb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:34:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 --2- 172.21.15.66:0/1883981137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759815a420 0x7f759806fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 -- 172.21.15.66:0/1883981137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75980b4a50 msgr2=0x7f759815cd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:34:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 --2- 172.21.15.66:0/1883981137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75980b4a50 0x7f759815cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.743+0000 7f7596ffd640 1 -- 172.21.15.66:0/1883981137 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75982014f0 con 0x7f75980cd4f0 2024-09-10T22:34:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f7596ffd640 1 --2- 172.21.15.66:0/1883981137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75980cd4f0 0x7f75980c74a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f759000c9b0 tx=0x7f759000ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f757ffff640 1 -- 172.21.15.66:0/1883981137 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590003d80 con 0x7f75980cd4f0 2024-09-10T22:34:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f047a7fc640 1 -- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f047c15e120 msgr2=0x7f047c1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f047a7fc640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f047c15e120 0x7f047c1544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f047a7fc640 1 -- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 msgr2=0x7f047c0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f047a7fc640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 0x7f047c0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f047a7fc640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c1f4990 con 0x7f047c0d1f90 2024-09-10T22:34:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.744+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75982017d0 con 0x7f75980cd4f0 2024-09-10T22:34:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.745+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7598201d90 con 0x7f75980cd4f0 2024-09-10T22:34:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.745+0000 7f757ffff640 1 -- 172.21.15.66:0/1883981137 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f759000a700 con 0x7f75980cd4f0 2024-09-10T22:34:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.745+0000 7f757ffff640 1 -- 172.21.15.66:0/1883981137 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759000ab50 con 0x7f75980cd4f0 2024-09-10T22:34:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.745+0000 7f047a7fc640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f047c0d1f90 0x7f047c153f80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0470031000 tx=0x7f04700053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.746+0000 7f757ffff640 1 -- 172.21.15.66:0/1883981137 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f7590005070 con 0x7f75980cd4f0 2024-09-10T22:34:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.746+0000 7f757ffff640 1 -- 172.21.15.66:0/1883981137 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f759001bc60 con 0x7f75980cd4f0 2024-09-10T22:34:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.746+0000 7f0463fff640 1 -- 172.21.15.66:0/3922141828 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470004050 con 0x7f047c0d1f90 2024-09-10T22:34:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.746+0000 7f0463fff640 1 -- 172.21.15.66:0/3922141828 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04700042e0 con 0x7f047c0d1f90 2024-09-10T22:34:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.747+0000 7f0463fff640 1 -- 172.21.15.66:0/3922141828 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470004a00 con 0x7f047c0d1f90 2024-09-10T22:34:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.748+0000 7f0481833640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c1f4c70 con 0x7f047c0d1f90 2024-09-10T22:34:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.748+0000 7f0463fff640 1 -- 172.21.15.66:0/3922141828 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f0470004ba0 con 0x7f047c0d1f90 2024-09-10T22:34:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.748+0000 7f0481833640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1f5230 con 0x7f047c0d1f90 2024-09-10T22:34:38.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.752+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7578005290 con 0x7f75980cd4f0 2024-09-10T22:34:38.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.753+0000 7f0463fff640 1 -- 172.21.15.66:0/3922141828 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0470003930 con 0x7f047c0d1f90 2024-09-10T22:34:38.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.754+0000 7f757ffff640 1 -- 172.21.15.66:0/1883981137 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f759001b400 con 0x7f75980cd4f0 2024-09-10T22:34:38.756 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.755+0000 7f0461ffb640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047c0b7770 con 0x7f047c0d1f90 2024-09-10T22:34:38.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.759+0000 7f0463fff640 1 -- 172.21.15.66:0/3922141828 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f0470040030 con 0x7f047c0d1f90 2024-09-10T22:34:38.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 -- 172.21.15.66:0/3785643620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70008b940 msgr2=0x7fe70016b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 --2- 172.21.15.66:0/3785643620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70008b940 0x7fe70016b9d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7000b2010 tx=0x7fe6fc02b590 comp rx=0 tx=0).stop 2024-09-10T22:34:38.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 -- 172.21.15.66:0/3785643620 shutdown_connections 2024-09-10T22:34:38.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 --2- 172.21.15.66:0/3785643620 >> v2:172.21.15.66:3300/0 conn(0x7fe70006ebd0 0x7fe700072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 --2- 172.21.15.66:0/3785643620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe70008b940 0x7fe70016b9d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 -- 172.21.15.66:0/3785643620 >> 172.21.15.66:0/3785643620 conn(0x7fe70006d230 msgr2=0x7fe7000b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.777+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fc4540aeda0 con 0x7fc4540cdd40 2024-09-10T22:34:38.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.784+0000 7fc450ff9640 1 -- 172.21.15.66:0/2674418539 <== mon.2 v2:172.21.15.94:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7fc44c020030 con 0x7fc4540cdd40 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 -- 172.21.15.66:0/3785643620 shutdown_connections 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.772+0000 7fe70605c640 1 -- 172.21.15.66:0/3785643620 wait complete. 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.774+0000 7fe70605c640 1 Processor -- start 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.774+0000 7fe70605c640 1 -- start start 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.776+0000 7fe70605c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70006ebd0 0x7fe700164ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.776+0000 7fe70605c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe700070780 0x7fe700089380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.776+0000 7fe70605c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70008a390 0x7fe7000898c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.776+0000 7fe70605c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe700072b20 con 0x7fe70006ebd0 2024-09-10T22:34:38.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.776+0000 7fe70605c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe7000729a0 con 0x7fe700070780 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.776+0000 7fe70605c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe700072ca0 con 0x7fe70008a390 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.782+0000 7fe704859640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe700070780 0x7fe700089380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.782+0000 7fe704859640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe700070780 0x7fe700089380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:48440/0 (socket says 172.21.15.66:48440) 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.782+0000 7fe704859640 1 -- 172.21.15.66:0/4163661487 learned_addr learned my addr 172.21.15.66:0/4163661487 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.782+0000 7fe70585b640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70008a390 0x7fe7000898c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.782+0000 7fe70505a640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70006ebd0 0x7fe700164ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.783+0000 7fe704859640 1 -- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70008a390 msgr2=0x7fe7000898c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.783+0000 7fe704859640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70008a390 0x7fe7000898c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.783+0000 7fe704859640 1 -- 172.21.15.66:0/4163661487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70006ebd0 msgr2=0x7fe700164ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.784+0000 7fe704859640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70006ebd0 0x7fe700164ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.784+0000 7fe704859640 1 -- 172.21.15.66:0/4163661487 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70008ab80 con 0x7fe700070780 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.784+0000 7fe70505a640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70006ebd0 0x7fe700164ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.784+0000 7fe704859640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe700070780 0x7fe700089380 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800cc50 tx=0x7fe6f800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.785+0000 7fe6f5ffb640 1 -- 172.21.15.66:0/4163661487 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8016020 con 0x7fe700070780 2024-09-10T22:34:38.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.785+0000 7fe6f5ffb640 1 -- 172.21.15.66:0/4163661487 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f8012420 con 0x7fe700070780 2024-09-10T22:34:38.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.785+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe700153d70 con 0x7fe700070780 2024-09-10T22:34:38.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.785+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe700154330 con 0x7fe700070780 2024-09-10T22:34:38.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.785+0000 7fe6f5ffb640 1 -- 172.21.15.66:0/4163661487 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800ce60 con 0x7fe700070780 2024-09-10T22:34:38.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.786+0000 7fe6f5ffb640 1 -- 172.21.15.66:0/4163661487 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fe6f8012cc0 con 0x7fe700070780 2024-09-10T22:34:38.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.786+0000 7fe6f5ffb640 1 -- 172.21.15.66:0/4163661487 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe6f801a720 con 0x7fe700070780 2024-09-10T22:34:38.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.789+0000 7f87f4a58640 1 -- 172.21.15.66:0/916184416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87d8003a00 msgr2=0x7f87d8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.789+0000 7f87f4a58640 1 --2- 172.21.15.66:0/916184416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87d8003a00 0x7f87d8003e00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f87e0001f00 tx=0x7f87d8005830 comp rx=0 tx=0).stop 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cdd40 msgr2=0x7fc4540c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cdd40 0x7fc4540c4fe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00cbd0 tx=0x7fc44c00a430 comp rx=0 tx=0).stop 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 shutdown_connections 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4541506e0 0x7fc4540b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc4540cdd40 0x7fc4540c4fe0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 --2- 172.21.15.66:0/2674418539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc4540cd960 0x7fc4540aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 >> 172.21.15.66:0/2674418539 conn(0x7fc4540b16d0 msgr2=0x7fc4540cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 shutdown_connections 2024-09-10T22:34:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.793+0000 7fc45a808640 1 -- 172.21.15.66:0/2674418539 wait complete. 2024-09-10T22:34:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.794+0000 7f87f4a58640 1 -- 172.21.15.66:0/916184416 shutdown_connections 2024-09-10T22:34:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.794+0000 7f87f4a58640 1 --2- 172.21.15.66:0/916184416 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87d8003a00 0x7f87d8003e00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.794+0000 7f87f4a58640 1 -- 172.21.15.66:0/916184416 >> 172.21.15.66:0/916184416 conn(0x7f87e801d770 msgr2=0x7f87e8054560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.794+0000 7f87f4a58640 1 -- 172.21.15.66:0/916184416 shutdown_connections 2024-09-10T22:34:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.794+0000 7f87f4a58640 1 -- 172.21.15.66:0/916184416 wait complete. 2024-09-10T22:34:38.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.794+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e0005290 con 0x7fe700070780 2024-09-10T22:34:38.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.797+0000 7fe6f5ffb640 1 -- 172.21.15.66:0/4163661487 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fe6f8020020 con 0x7fe700070780 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.797+0000 7f87f4a58640 1 Processor -- start 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.797+0000 7f87f4a58640 1 -- start start 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.798+0000 7f87f4a58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d8003a00 0x7f87e81795e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.799+0000 7f87f4a58640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87e8049990 0x7f87e801f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.799+0000 7f87eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d8003a00 0x7f87e81795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.799+0000 7f87eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d8003a00 0x7f87e81795e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:48466/0 (socket says 172.21.15.66:48466) 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.799+0000 7f87eed9d640 1 -- 172.21.15.66:0/2857307982 learned_addr learned my addr 172.21.15.66:0/2857307982 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.799+0000 7f87ee59c640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87e8049990 0x7f87e801f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.799+0000 7f87f4a58640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e80de7f0 0x7f87e801fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.804+0000 7f87f4a58640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80e3cd0 con 0x7f87e80de7f0 2024-09-10T22:34:38.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.804+0000 7f87f4a58640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80e3b50 con 0x7f87d8003a00 2024-09-10T22:34:38.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.804+0000 7f87f4a58640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f87e80e3e50 con 0x7f87e8049990 2024-09-10T22:34:38.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.804+0000 7f87ef59e640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e80de7f0 0x7f87e801fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.806+0000 7f87eed9d640 1 -- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87e8049990 msgr2=0x7f87e801f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.806+0000 7f87eed9d640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87e8049990 0x7f87e801f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.806+0000 7f87eed9d640 1 -- 172.21.15.66:0/2857307982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e80de7f0 msgr2=0x7f87e801fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.806+0000 7f87eed9d640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e80de7f0 0x7f87e801fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.806+0000 7f87eed9d640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8178b40 con 0x7f87d8003a00 2024-09-10T22:34:38.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.807+0000 7f87eed9d640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d8003a00 0x7f87e81795e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87e0001f00 tx=0x7f87e0001770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.811+0000 7f87d77fe640 1 -- 172.21.15.66:0/2857307982 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0044070 con 0x7f87d8003a00 2024-09-10T22:34:38.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.811+0000 7f87d77fe640 1 -- 172.21.15.66:0/2857307982 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87e0001cd0 con 0x7f87d8003a00 2024-09-10T22:34:38.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.811+0000 7f87d77fe640 1 -- 172.21.15.66:0/2857307982 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e002dac0 con 0x7f87d8003a00 2024-09-10T22:34:38.813 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.814+0000 7f87f4a58640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e8049e20 con 0x7f87d8003a00 2024-09-10T22:34:38.818 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.815+0000 7f87f4a58640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81790c0 con 0x7f87d8003a00 2024-09-10T22:34:38.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.818+0000 7f87d77fe640 1 -- 172.21.15.66:0/2857307982 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f87e002dc60 con 0x7f87d8003a00 2024-09-10T22:34:38.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.818+0000 7f87d77fe640 1 -- 172.21.15.66:0/2857307982 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f87e00368a0 con 0x7f87d8003a00 2024-09-10T22:34:38.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.819+0000 7faeb0100640 1 -- 172.21.15.66:0/22169994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea80cfc80 msgr2=0x7faea815a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.820 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.819+0000 7faeb0100640 1 --2- 172.21.15.66:0/22169994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea80cfc80 0x7faea815a390 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae98005fd0 tx=0x7fae9802a740 comp rx=0 tx=0).stop 2024-09-10T22:34:38.822 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.821+0000 7f87d57fa640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e8036130 con 0x7f87d8003a00 2024-09-10T22:34:38.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 -- 172.21.15.66:0/707167439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26641042e0 msgr2=0x7f2664104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 --2- 172.21.15.66:0/707167439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26641042e0 0x7f2664104770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f266c000b60 tx=0x7f266c094e40 comp rx=0 tx=0).stop 2024-09-10T22:34:38.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 -- 172.21.15.66:0/707167439 shutdown_connections 2024-09-10T22:34:38.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 --2- 172.21.15.66:0/707167439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26641042e0 0x7f2664104770 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 --2- 172.21.15.66:0/707167439 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2664100480 0x7f2664100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 --2- 172.21.15.66:0/707167439 >> v2:172.21.15.66:3300/0 conn(0x7f2664054c70 0x7f2664033ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.826 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 -- 172.21.15.66:0/707167439 >> 172.21.15.66:0/707167439 conn(0x7f266401d770 msgr2=0x7f2664054560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 -- 172.21.15.66:0/707167439 shutdown_connections 2024-09-10T22:34:38.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.825+0000 7f2670c23640 1 -- 172.21.15.66:0/707167439 wait complete. 2024-09-10T22:34:38.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.827+0000 7f2670c23640 1 Processor -- start 2024-09-10T22:34:38.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.826+0000 7f3f9359e640 1 -- 172.21.15.66:0/3883946349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f800052e0 msgr2=0x7f3f800141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.826+0000 7f3f9159a640 1 -- 172.21.15.66:0/3883946349 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00c260 con 0x7f3f800052e0 2024-09-10T22:34:38.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.826+0000 7f3f9359e640 1 --2- 172.21.15.66:0/3883946349 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f800052e0 0x7f3f800141d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00c610 tx=0x7f3f80005830 comp rx=0 tx=0).stop 2024-09-10T22:34:38.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.826+0000 7f3f92d9d640 1 -- 172.21.15.66:0/3883946349 reap_dead start 2024-09-10T22:34:38.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.826+0000 7f3f9359e640 1 -- 172.21.15.66:0/3883946349 shutdown_connections 2024-09-10T22:34:38.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.826+0000 7f3f9359e640 1 -- 172.21.15.66:0/3883946349 >> 172.21.15.66:0/3883946349 conn(0x7f3f9406d230 msgr2=0x7f3f940b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.831 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.830+0000 7f3f9359e640 1 -- 172.21.15.66:0/3883946349 shutdown_connections 2024-09-10T22:34:38.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.831+0000 7faeb0100640 1 -- 172.21.15.66:0/22169994 shutdown_connections 2024-09-10T22:34:38.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.831+0000 7faeb0100640 1 --2- 172.21.15.66:0/22169994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea80cfc80 0x7faea815a390 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.831+0000 7faeb0100640 1 -- 172.21.15.66:0/22169994 >> 172.21.15.66:0/22169994 conn(0x7faea80b1f00 msgr2=0x7faea80b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.833+0000 7f87d77fe640 1 -- 172.21.15.66:0/2857307982 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f87e0036aa0 con 0x7f87d8003a00 2024-09-10T22:34:38.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.837+0000 7faeb0100640 1 -- 172.21.15.66:0/22169994 shutdown_connections 2024-09-10T22:34:38.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.837+0000 7faeb0100640 1 -- 172.21.15.66:0/22169994 wait complete. 2024-09-10T22:34:38.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7faeb0100640 1 Processor -- start 2024-09-10T22:34:38.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7faeb0100640 1 -- start start 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.836+0000 7f3f9359e640 1 -- 172.21.15.66:0/3883946349 wait complete. 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.838+0000 7f3f9359e640 1 Processor -- start 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.838+0000 7f3f9359e640 1 -- start start 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f94089fa0 0x7f3f94168ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f9408a380 0x7f3f94169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f942013b0 0x7f3f94201840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7f3f9359e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94070b10 con 0x7f3f942013b0 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7f3f9359e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94070990 con 0x7f3f9408a380 2024-09-10T22:34:38.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.839+0000 7f3f9359e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9408be60 con 0x7f3f94089fa0 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f9408a380 0x7f3f94169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f91d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f9408a380 0x7f3f94169530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:48478/0 (socket says 172.21.15.66:48478) 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f91d9b640 1 -- 172.21.15.66:0/815024577 learned_addr learned my addr 172.21.15.66:0/815024577 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f92d9d640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f942013b0 0x7f3f94201840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f92d9d640 1 -- 172.21.15.66:0/815024577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f94089fa0 msgr2=0x7f3f94168ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f92d9d640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f94089fa0 0x7f3f94168ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f92d9d640 1 -- 172.21.15.66:0/815024577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f9408a380 msgr2=0x7f3f94169530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f92d9d640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f9408a380 0x7f3f94169530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.840+0000 7f3f92d9d640 1 -- 172.21.15.66:0/815024577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f94165620 con 0x7f3f942013b0 2024-09-10T22:34:38.843 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.841+0000 7faeb0100640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faea80ccde0 0x7faea80b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.841+0000 7faeb0100640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea80cfc80 0x7faea815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.841+0000 7f3f92d9d640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f942013b0 0x7f3f94201840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c030000 tx=0x7f3f8c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.841+0000 7faeb0100640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea815e940 0x7faea815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7faeade9c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faea80ccde0 0x7faea80b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7f3f8affd640 1 -- 172.21.15.66:0/815024577 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c02b880 con 0x7f3f942013b0 2024-09-10T22:34:38.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7faeade9c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faea80ccde0 0x7faea80b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:40850/0 (socket says 172.21.15.66:40850) 2024-09-10T22:34:38.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7faeade9c640 1 -- 172.21.15.66:0/1415157485 learned_addr learned my addr 172.21.15.66:0/1415157485 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7faead69b640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea80cfc80 0x7faea815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.841+0000 7faeae69d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea815e940 0x7faea815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7f3f8affd640 1 -- 172.21.15.66:0/815024577 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f8c02ba20 con 0x7f3f942013b0 2024-09-10T22:34:38.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f94165900 con 0x7f3f942013b0 2024-09-10T22:34:38.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.842+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f94165ec0 con 0x7f3f942013b0 2024-09-10T22:34:38.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.843+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f94072460 con 0x7f3f942013b0 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.843+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faea80cd5b0 con 0x7faea815e940 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.844+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faea80cd430 con 0x7faea80cfc80 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.844+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7faea80cd730 con 0x7faea80ccde0 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.844+0000 7faeade9c640 1 -- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea80cfc80 msgr2=0x7faea815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.844+0000 7faeade9c640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea80cfc80 0x7faea815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.844+0000 7faeade9c640 1 -- 172.21.15.66:0/1415157485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea815e940 msgr2=0x7faea815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.844+0000 7faeade9c640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea815e940 0x7faea815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.847 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.844+0000 7faeade9c640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea815dd00 con 0x7faea80ccde0 2024-09-10T22:34:38.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.845+0000 7faeade9c640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faea80ccde0 0x7faea80b5640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faea000c770 tx=0x7faea000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.845+0000 7f2670c23640 1 -- start start 2024-09-10T22:34:38.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7fae9e7fc640 1 -- 172.21.15.66:0/1415157485 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0003b70 con 0x7faea80ccde0 2024-09-10T22:34:38.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7fae9e7fc640 1 -- 172.21.15.66:0/1415157485 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faea0003d10 con 0x7faea80ccde0 2024-09-10T22:34:38.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7fae9e7fc640 1 -- 172.21.15.66:0/1415157485 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea000aab0 con 0x7faea80ccde0 2024-09-10T22:34:38.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea80d0d90 con 0x7faea80ccde0 2024-09-10T22:34:38.848 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faea80c66f0 con 0x7faea80ccde0 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f2670c23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664054c70 0x7f26641a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f2670c23640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2664100480 0x7f26641a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f2670c23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26641042e0 0x7f2664101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f2670c23640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2664005de0 con 0x7f2664054c70 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f2670c23640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2664005c60 con 0x7f26641042e0 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f2670c23640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2664005f60 con 0x7f2664100480 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f266a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664054c70 0x7f26641a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f266a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664054c70 0x7f26641a7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57264/0 (socket says 172.21.15.66:57264) 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.846+0000 7f266a59c640 1 -- 172.21.15.66:0/27824945 learned_addr learned my addr 172.21.15.66:0/27824945 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7f266a59c640 1 -- 172.21.15.66:0/27824945 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2664100480 msgr2=0x7f26641a1fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:34:38.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7f266a59c640 1 --2- 172.21.15.66:0/27824945 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2664100480 0x7f26641a1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7f266a59c640 1 -- 172.21.15.66:0/27824945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26641042e0 msgr2=0x7f2664101ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:34:38.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7f266a59c640 1 --2- 172.21.15.66:0/27824945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26641042e0 0x7f2664101ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7f266a59c640 1 -- 172.21.15.66:0/27824945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2664103820 con 0x7f2664054c70 2024-09-10T22:34:38.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.847+0000 7f266a59c640 1 --2- 172.21.15.66:0/27824945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664054c70 0x7f26641a7f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f265c00b260 tx=0x7f265c00b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:38.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.850+0000 7fae9e7fc640 1 -- 172.21.15.66:0/1415157485 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7faea000ac50 con 0x7faea80ccde0 2024-09-10T22:34:38.852 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.850+0000 7fae9e7fc640 1 -- 172.21.15.66:0/1415157485 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faea0012990 con 0x7faea80ccde0 2024-09-10T22:34:38.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.853+0000 7f265affd640 1 -- 172.21.15.66:0/27824945 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c004220 con 0x7f2664054c70 2024-09-10T22:34:38.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.853+0000 7f265affd640 1 -- 172.21.15.66:0/27824945 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f265c0112e0 con 0x7f2664054c70 2024-09-10T22:34:38.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.853+0000 7f265affd640 1 -- 172.21.15.66:0/27824945 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c011710 con 0x7f2664054c70 2024-09-10T22:34:38.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.855+0000 7f5c7dffb640 1 -- 172.21.15.66:0/3780158152 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f5c78005850 con 0x7f5c980b0b10 2024-09-10T22:34:38.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.855+0000 7f3f8affd640 1 -- 172.21.15.66:0/815024577 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c02bdf0 con 0x7f3f942013b0 2024-09-10T22:34:38.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.855+0000 7f3f8affd640 1 -- 172.21.15.66:0/815024577 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3f8c034800 con 0x7f3f942013b0 2024-09-10T22:34:38.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.855+0000 7f3f8affd640 1 -- 172.21.15.66:0/815024577 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f8c03ecf0 con 0x7f3f942013b0 2024-09-10T22:34:38.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.859+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2664102780 con 0x7f2664054c70 2024-09-10T22:34:38.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.859+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2664102d40 con 0x7f2664054c70 2024-09-10T22:34:38.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.860+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae90005290 con 0x7faea80ccde0 2024-09-10T22:34:38.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.860+0000 7f3f8affd640 1 -- 172.21.15.66:0/815024577 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f3f8c043030 con 0x7f3f942013b0 2024-09-10T22:34:38.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.861+0000 7f265affd640 1 -- 172.21.15.66:0/27824945 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f265c0053b0 con 0x7f2664054c70 2024-09-10T22:34:38.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.865+0000 7f265affd640 1 -- 172.21.15.66:0/27824945 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f265c003930 con 0x7f2664054c70 2024-09-10T22:34:38.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.866+0000 7f5c7ffff640 1 -- 172.21.15.66:0/3780158152 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f5c9003d480 con 0x7f5c980b0b10 2024-09-10T22:34:38.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.859+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f566c005850 con 0x7f567c0d25e0 2024-09-10T22:34:38.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.864+0000 7fae9e7fc640 1 -- 172.21.15.66:0/1415157485 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7faea00050d0 con 0x7faea80ccde0 2024-09-10T22:34:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.868+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266400dbf0 con 0x7f2664054c70 2024-09-10T22:34:38.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.872+0000 7f265affd640 1 -- 172.21.15.66:0/27824945 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f265c021030 con 0x7f2664054c70 2024-09-10T22:34:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.875+0000 7f567affd640 1 -- 172.21.15.66:0/3351967847 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f56840bea00 con 0x7f567c0d25e0 2024-09-10T22:34:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.875+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/3780158152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980b0b10 msgr2=0x7f5c981f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.875+0000 7f5c9cb8a640 1 --2- 172.21.15.66:0/3780158152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980b0b10 0x7f5c981f5050 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c680 tx=0x7f5c90001aa0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/3780158152 shutdown_connections 2024-09-10T22:34:38.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f5c9cb8a640 1 --2- 172.21.15.66:0/3780158152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980cc340 0x7f5c980cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f5c9cb8a640 1 --2- 172.21.15.66:0/3780158152 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980b0b10 0x7f5c981f5050 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f5c9cb8a640 1 --2- 172.21.15.66:0/3780158152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c9815add0 0x7f5c980af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.877 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/3780158152 >> 172.21.15.66:0/3780158152 conn(0x7f5c9806d230 msgr2=0x7f5c9806eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c0d25e0 msgr2=0x7f567c048570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 --2- 172.21.15.66:0/3351967847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c0d25e0 0x7f567c048570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f56840668e0 tx=0x7f56840b7ab0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 shutdown_connections 2024-09-10T22:34:38.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 --2- 172.21.15.66:0/3351967847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c0d25e0 0x7f567c048570 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 --2- 172.21.15.66:0/3351967847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f567c0e69e0 0x7f567c048030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 --2- 172.21.15.66:0/3351967847 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f567c04eb50 0x7f567c04a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 >> 172.21.15.66:0/3351967847 conn(0x7f567c01d7a0 msgr2=0x7f567c0e5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/3780158152 shutdown_connections 2024-09-10T22:34:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f5c9cb8a640 1 -- 172.21.15.66:0/3780158152 wait complete. 2024-09-10T22:34:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 shutdown_connections 2024-09-10T22:34:38.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.876+0000 7f568c7ca640 1 -- 172.21.15.66:0/3351967847 wait complete. 2024-09-10T22:34:38.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.884+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f425c005580 con 0x7f427c16b9f0 2024-09-10T22:34:38.895 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.894+0000 7f4271ffb640 1 -- 172.21.15.66:0/2388326192 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7f4274005070 con 0x7f427c16b9f0 2024-09-10T22:34:38.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.904+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c16b9f0 msgr2=0x7f427c073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.904+0000 7f42834dd640 1 --2- 172.21.15.66:0/2388326192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c16b9f0 0x7f427c073490 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f427400cb10 tx=0x7f427400a060 comp rx=0 tx=0).stop 2024-09-10T22:34:38.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.904+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 shutdown_connections 2024-09-10T22:34:38.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.904+0000 7f42834dd640 1 --2- 172.21.15.66:0/2388326192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f427c201fc0 0x7f427c0739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.904+0000 7f42834dd640 1 --2- 172.21.15.66:0/2388326192 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c16b9f0 0x7f427c073490 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.905+0000 7f42834dd640 1 --2- 172.21.15.66:0/2388326192 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f427c0b41f0 0x7f427c08acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.905+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 >> 172.21.15.66:0/2388326192 conn(0x7f427c0cfaa0 msgr2=0x7f427c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.905+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 shutdown_connections 2024-09-10T22:34:38.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.905+0000 7f42834dd640 1 -- 172.21.15.66:0/2388326192 wait complete. 2024-09-10T22:34:38.907 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.907 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.932+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f7578005580 con 0x7f75980cd4f0 2024-09-10T22:34:38.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.942+0000 7f0461ffb640 1 -- 172.21.15.66:0/3922141828 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f047c050fc0 con 0x7f047c0d1f90 2024-09-10T22:34:38.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.943+0000 7f757ffff640 1 -- 172.21.15.66:0/1883981137 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7f759000acf0 con 0x7f75980cd4f0 2024-09-10T22:34:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.945+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75980cd4f0 msgr2=0x7f75980c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.945+0000 7f759d852640 1 --2- 172.21.15.66:0/1883981137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75980cd4f0 0x7f75980c74a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f759000c9b0 tx=0x7f759000ce70 comp rx=0 tx=0).stop 2024-09-10T22:34:38.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.946+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 shutdown_connections 2024-09-10T22:34:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.946+0000 7f759d852640 1 --2- 172.21.15.66:0/1883981137 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75980b4a50 0x7f759815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.946+0000 7f759d852640 1 --2- 172.21.15.66:0/1883981137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f759815a420 0x7f759806fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.946+0000 7f759d852640 1 --2- 172.21.15.66:0/1883981137 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f75980cd4f0 0x7f75980c74a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.946+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 >> 172.21.15.66:0/1883981137 conn(0x7f75980b1f00 msgr2=0x7f75980b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.947+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 shutdown_connections 2024-09-10T22:34:38.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.947+0000 7f759d852640 1 -- 172.21.15.66:0/1883981137 wait complete. 2024-09-10T22:34:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.951+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fe6e0005580 con 0x7fe700070780 2024-09-10T22:34:38.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.951+0000 7f0463fff640 1 -- 172.21.15.66:0/3922141828 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v13)=0 v13) ==== 122+0+0 (secure 0 0 0) 0x7f0470003b30 con 0x7f047c0d1f90 2024-09-10T22:34:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.953+0000 7f0461ffb640 1 -- 172.21.15.66:0/3922141828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f047c0d1f90 msgr2=0x7f047c153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.954 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.953+0000 7f0461ffb640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f047c0d1f90 0x7f047c153f80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0470031000 tx=0x7f04700053b0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.954+0000 7f0461ffb640 1 -- 172.21.15.66:0/3922141828 shutdown_connections 2024-09-10T22:34:38.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.954+0000 7f0461ffb640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f047c15e120 0x7f047c1544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.954+0000 7f0461ffb640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f047c0d1f90 0x7f047c153f80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.954+0000 7f0461ffb640 1 --2- 172.21.15.66:0/3922141828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f047c0d10d0 0x7f047c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.954+0000 7f0461ffb640 1 -- 172.21.15.66:0/3922141828 >> 172.21.15.66:0/3922141828 conn(0x7f047c0d3f10 msgr2=0x7f047c0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.954+0000 7f0461ffb640 1 -- 172.21.15.66:0/3922141828 shutdown_connections 2024-09-10T22:34:38.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.954+0000 7f0461ffb640 1 -- 172.21.15.66:0/3922141828 wait complete. 2024-09-10T22:34:38.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.964+0000 7fe6f5ffb640 1 -- 172.21.15.66:0/4163661487 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v14)=0 v14) ==== 119+0+0 (secure 0 0 0) 0x7fe6f801a9d0 con 0x7fe700070780 2024-09-10T22:34:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.966+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe700070780 msgr2=0x7fe700089380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.966+0000 7fe70605c640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe700070780 0x7fe700089380 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800cc50 tx=0x7fe6f800a430 comp rx=0 tx=0).stop 2024-09-10T22:34:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.966+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 shutdown_connections 2024-09-10T22:34:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.966+0000 7fe70605c640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe70008a390 0x7fe7000898c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.966+0000 7fe70605c640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe700070780 0x7fe700089380 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.966+0000 7fe70605c640 1 --2- 172.21.15.66:0/4163661487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe70006ebd0 0x7fe700164ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.966+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 >> 172.21.15.66:0/4163661487 conn(0x7fe70006d230 msgr2=0x7fe7000b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.967+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 shutdown_connections 2024-09-10T22:34:38.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.968+0000 7fe70605c640 1 -- 172.21.15.66:0/4163661487 wait complete. 2024-09-10T22:34:38.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.970+0000 7f116e117640 1 -- 172.21.15.66:0/3512422871 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f1148005850 con 0x7f116808ba70 2024-09-10T22:34:38.974 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.974 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.976+0000 7f114ffff640 1 -- 172.21.15.66:0/3512422871 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f11580035f0 con 0x7f116808ba70 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 -- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f116808ba70 msgr2=0x7f11680aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f116808ba70 0x7f11680aece0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f115800c480 tx=0x7f115800c940 comp rx=0 tx=0).stop 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 -- 172.21.15.66:0/3512422871 shutdown_connections 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1168200d70 0x7f11680ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f116816b9f0 0x7f1168200830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 --2- 172.21.15.66:0/3512422871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f116808ba70 0x7f11680aece0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 -- 172.21.15.66:0/3512422871 >> 172.21.15.66:0/3512422871 conn(0x7f11680cfaa0 msgr2=0x7f1168072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.978+0000 7f114dffb640 1 -- 172.21.15.66:0/3512422871 shutdown_connections 2024-09-10T22:34:38.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:38.979+0000 7f114dffb640 1 -- 172.21.15.66:0/3512422871 wait complete. 2024-09-10T22:34:38.985 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.985 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:34:38.997 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi066') 2024-09-10T22:34:39.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.003+0000 7f87d57fa640 1 -- 172.21.15.66:0/2857307982 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f87e8033520 con 0x7f87d8003a00 2024-09-10T22:34:39.009 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.008+0000 7f87d77fe640 1 -- 172.21.15.66:0/2857307982 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) ==== 120+0+0 (secure 0 0 0) 0x7f87e0036c80 con 0x7f87d8003a00 2024-09-10T22:34:39.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 -- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d8003a00 msgr2=0x7f87e81795e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d8003a00 0x7f87e81795e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87e0001f00 tx=0x7f87e0001770 comp rx=0 tx=0).stop 2024-09-10T22:34:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 -- 172.21.15.66:0/2857307982 shutdown_connections 2024-09-10T22:34:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f87e80de7f0 0x7f87e801fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f87e8049990 0x7f87e801f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 --2- 172.21.15.66:0/2857307982 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87d8003a00 0x7f87e81795e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 -- 172.21.15.66:0/2857307982 >> 172.21.15.66:0/2857307982 conn(0x7f87e801d770 msgr2=0x7f87e8052c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 -- 172.21.15.66:0/2857307982 shutdown_connections 2024-09-10T22:34:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.009+0000 7f87d57fa640 1 -- 172.21.15.66:0/2857307982 wait complete. 2024-09-10T22:34:39.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.023+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fae90005580 con 0x7faea80ccde0 2024-09-10T22:34:39.024 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi066') 2024-09-10T22:34:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.028+0000 7fae9e7fc640 1 -- 172.21.15.66:0/1415157485 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v17)=0 v17) ==== 125+0+0 (secure 0 0 0) 0x7faea0012be0 con 0x7faea80ccde0 2024-09-10T22:34:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faea80ccde0 msgr2=0x7faea80b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faea80ccde0 0x7faea80b5640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faea000c770 tx=0x7faea000cc30 comp rx=0 tx=0).stop 2024-09-10T22:34:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 shutdown_connections 2024-09-10T22:34:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faea815e940 0x7faea815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faea80cfc80 0x7faea815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 --2- 172.21.15.66:0/1415157485 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faea80ccde0 0x7faea80b5640 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 >> 172.21.15.66:0/1415157485 conn(0x7faea80b1f00 msgr2=0x7faea81596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 shutdown_connections 2024-09-10T22:34:39.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.029+0000 7faeb0100640 1 -- 172.21.15.66:0/1415157485 wait complete. 2024-09-10T22:34:39.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.033+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f3f940b04d0 con 0x7f3f942013b0 2024-09-10T22:34:39.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.036+0000 7f3f8affd640 1 -- 172.21.15.66:0/815024577 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f3f8c00c2d0 con 0x7f3f942013b0 2024-09-10T22:34:39.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f942013b0 msgr2=0x7f3f94201840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f942013b0 0x7f3f94201840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c030000 tx=0x7f3f8c003f40 comp rx=0 tx=0).stop 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 shutdown_connections 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f942013b0 0x7f3f94201840 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f9408a380 0x7f3f94169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 --2- 172.21.15.66:0/815024577 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f94089fa0 0x7f3f94168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 >> 172.21.15.66:0/815024577 conn(0x7f3f9406d230 msgr2=0x7f3f94162d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 shutdown_connections 2024-09-10T22:34:39.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.037+0000 7f3f9359e640 1 -- 172.21.15.66:0/815024577 wait complete. 2024-09-10T22:34:39.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.039+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f2664032de0 con 0x7f2664054c70 2024-09-10T22:34:39.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.041+0000 7f265affd640 1 -- 172.21.15.66:0/27824945 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f265c0047c0 con 0x7f2664054c70 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.042+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664054c70 msgr2=0x7f26641a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.042+0000 7f2670c23640 1 --2- 172.21.15.66:0/27824945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664054c70 0x7f26641a7f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f265c00b260 tx=0x7f265c00b720 comp rx=0 tx=0).stop 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.042+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 shutdown_connections 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.042+0000 7f2670c23640 1 --2- 172.21.15.66:0/27824945 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26641042e0 0x7f2664101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.042+0000 7f2670c23640 1 --2- 172.21.15.66:0/27824945 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2664100480 0x7f26641a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.042+0000 7f2670c23640 1 --2- 172.21.15.66:0/27824945 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2664054c70 0x7f26641a7f10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.042+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 >> 172.21.15.66:0/27824945 conn(0x7f266401d770 msgr2=0x7f26640099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.043+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 shutdown_connections 2024-09-10T22:34:39.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.043+0000 7f2670c23640 1 -- 172.21.15.66:0/27824945 wait complete. 2024-09-10T22:34:39.057 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi066') 2024-09-10T22:34:39.057 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi066') 2024-09-10T22:34:39.058 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T22:34:39.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T22:34:39.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.202+0000 7ff94d3db640 1 -- 172.21.15.66:0/2544388022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 msgr2=0x7ff9480d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.202+0000 7ff94d3db640 1 --2- 172.21.15.66:0/2544388022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480d2420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff940005fd0 tx=0x7ff94002a980 comp rx=0 tx=0).stop 2024-09-10T22:34:39.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.203+0000 7ff94d3db640 1 -- 172.21.15.66:0/2544388022 shutdown_connections 2024-09-10T22:34:39.204 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.203+0000 7ff94d3db640 1 --2- 172.21.15.66:0/2544388022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480d2420 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.203+0000 7ff94d3db640 1 --2- 172.21.15.66:0/2544388022 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9480d10d0 0x7ff9480d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.203+0000 7ff94d3db640 1 -- 172.21.15.66:0/2544388022 >> 172.21.15.66:0/2544388022 conn(0x7ff9480d3f10 msgr2=0x7ff9480d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.203+0000 7ff94d3db640 1 -- 172.21.15.66:0/2544388022 shutdown_connections 2024-09-10T22:34:39.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.203+0000 7ff94d3db640 1 -- 172.21.15.66:0/2544388022 wait complete. 2024-09-10T22:34:39.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.204+0000 7ff94d3db640 1 Processor -- start 2024-09-10T22:34:39.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.204+0000 7ff94d3db640 1 -- start start 2024-09-10T22:34:39.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff94d3db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9480d10d0 0x7ff9480b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:39.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff94d3db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:39.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff94d3db640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff94815e120 0x7ff9480cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:39.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff94d3db640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff9480b70f0 con 0x7ff9480d1f90 2024-09-10T22:34:39.207 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff94d3db640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff9480b6f70 con 0x7ff9480d10d0 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff94d3db640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff9480b7270 con 0x7ff94815e120 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57280/0 (socket says 172.21.15.66:57280) 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff946ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9480d10d0 0x7ff9480b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 -- 172.21.15.66:0/3524627655 learned_addr learned my addr 172.21.15.66:0/3524627655 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff9477fe640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff94815e120 0x7ff9480cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 -- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff94815e120 msgr2=0x7ff9480cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff94815e120 0x7ff9480cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 -- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9480d10d0 msgr2=0x7ff9480b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9480d10d0 0x7ff9480b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 -- 172.21.15.66:0/3524627655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9480c8a60 con 0x7ff9480d1f90 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff946ffd640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9480d10d0 0x7ff9480b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:39.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93ffff640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480c8120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d980 tx=0x7ff93000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93f7fe640 1 -- 172.21.15.66:0/3524627655 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930017070 con 0x7ff9480d1f90 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93f7fe640 1 -- 172.21.15.66:0/3524627655 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff930009d20 con 0x7ff9480d1f90 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff93f7fe640 1 -- 172.21.15.66:0/3524627655 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9300106c0 con 0x7ff9480d1f90 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.205+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9480c97c0 con 0x7ff9480d1f90 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.206+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9480c8660 con 0x7ff9480d1f90 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.206+0000 7ff93f7fe640 1 -- 172.21.15.66:0/3524627655 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff930010860 con 0x7ff9480d1f90 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.206+0000 7ff93f7fe640 1 -- 172.21.15.66:0/3524627655 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff93000f3c0 con 0x7ff9480d1f90 2024-09-10T22:34:39.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.206+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9480b7740 con 0x7ff9480d1f90 2024-09-10T22:34:39.210 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.208+0000 7ff93f7fe640 1 -- 172.21.15.66:0/3524627655 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7ff930010d60 con 0x7ff9480d1f90 2024-09-10T22:34:39.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.276+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff9480cad80 con 0x7ff9480d1f90 2024-09-10T22:34:39.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.277+0000 7ff93f7fe640 1 -- 172.21.15.66:0/3524627655 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ff93001c450 con 0x7ff9480d1f90 2024-09-10T22:34:39.278 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 msgr2=0x7ff9480c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480c8120 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff93000d980 tx=0x7ff93000de40 comp rx=0 tx=0).stop 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 shutdown_connections 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff94815e120 0x7ff9480cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9480d1f90 0x7ff9480c8120 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 --2- 172.21.15.66:0/3524627655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9480d10d0 0x7ff9480b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 >> 172.21.15.66:0/3524627655 conn(0x7ff9480d3f10 msgr2=0x7ff9480b8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 shutdown_connections 2024-09-10T22:34:39.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:39.278+0000 7ff94d3db640 1 -- 172.21.15.66:0/3524627655 wait complete. 2024-09-10T22:34:39.290 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T22:34:39.290 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T22:34:39.290 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:39.290 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T22:34:39.316 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:39.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T22:34:39.382 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:39.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T22:34:39.447 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:34:39.448 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T22:34:39.514 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:39.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T22:34:39.540 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:39.540 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T22:34:39.605 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:39.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T22:34:39.669 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:34:39.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T22:34:39.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 418ab0eb-5cba-46fb-a4ce-f29a4be053b1 0 2024-09-10T22:34:39.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 -- 172.21.15.94:0/677456223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8151960 msgr2=0x7f1ca80cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 --2- 172.21.15.94:0/677456223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8151960 0x7f1ca80cb270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009830 tx=0x7f1ca402a510 comp rx=0 tx=0).stop 2024-09-10T22:34:39.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 -- 172.21.15.94:0/677456223 shutdown_connections 2024-09-10T22:34:39.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 --2- 172.21.15.94:0/677456223 >> v2:172.21.15.94:3300/0 conn(0x7f1ca8073f40 0x7f1ca806fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 --2- 172.21.15.94:0/677456223 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8151960 0x7f1ca80cb270 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 -- 172.21.15.94:0/677456223 >> 172.21.15.94:0/677456223 conn(0x7f1ca80b16d0 msgr2=0x7f1ca80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 -- 172.21.15.94:0/677456223 shutdown_connections 2024-09-10T22:34:39.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.876+0000 7f1caf233640 1 -- 172.21.15.94:0/677456223 wait complete. 2024-09-10T22:34:39.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.877+0000 7f1caf233640 1 Processor -- start 2024-09-10T22:34:39.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.877+0000 7f1caf233640 1 -- start start 2024-09-10T22:34:39.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1caf233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca806f690 0x7f1ca81f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:39.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1caf233640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8073f40 0x7f1ca814f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1caf233640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca8151960 0x7f1ca814f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1caf233640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8151ec0 con 0x7f1ca8151960 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1caf233640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8151d40 con 0x7f1ca806f690 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1caf233640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca815ed60 con 0x7f1ca8073f40 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1cacfcf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca806f690 0x7f1ca81f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9ffff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8073f40 0x7f1ca814f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1cacfcf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca806f690 0x7f1ca81f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46618/0 (socket says 172.21.15.94:46618) 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1cacfcf640 1 -- 172.21.15.94:0/304372824 learned_addr learned my addr 172.21.15.94:0/304372824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1cad7d0640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca8151960 0x7f1ca814f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9ffff640 1 -- 172.21.15.94:0/304372824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca806f690 msgr2=0x7f1ca81f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9ffff640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca806f690 0x7f1ca81f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9ffff640 1 -- 172.21.15.94:0/304372824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca8151960 msgr2=0x7f1ca814f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9ffff640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca8151960 0x7f1ca814f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9ffff640 1 -- 172.21.15.94:0/304372824 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca81f2fb0 con 0x7f1ca8073f40 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9ffff640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8073f40 0x7f1ca814f4b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c3c0 tx=0x7f1c9800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9d7fa640 1 -- 172.21.15.94:0/304372824 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98017070 con 0x7f1ca8073f40 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.878+0000 7f1c9d7fa640 1 -- 172.21.15.94:0/304372824 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c980053c0 con 0x7f1ca8073f40 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.879+0000 7f1c9d7fa640 1 -- 172.21.15.94:0/304372824 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98003850 con 0x7f1ca8073f40 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.879+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca81f21f0 con 0x7f1ca8073f40 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.879+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca81f27b0 con 0x7f1ca8073f40 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.879+0000 7f1c9d7fa640 1 -- 172.21.15.94:0/304372824 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1c980039f0 con 0x7f1ca8073f40 2024-09-10T22:34:39.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.879+0000 7f1c9d7fa640 1 -- 172.21.15.94:0/304372824 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f1c9801c780 con 0x7f1ca8073f40 2024-09-10T22:34:39.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.879+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca815f280 con 0x7f1ca8073f40 2024-09-10T22:34:39.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.881+0000 7f1c9d7fa640 1 -- 172.21.15.94:0/304372824 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1c98021080 con 0x7f1ca8073f40 2024-09-10T22:34:39.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.952+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd new", "uuid": "418ab0eb-5cba-46fb-a4ce-f29a4be053b1", "id": 0} v 0) -- 0x7f1ca81f6500 con 0x7f1ca8073f40 2024-09-10T22:34:39.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.955+0000 7f1c9d7fa640 1 -- 172.21.15.94:0/304372824 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "418ab0eb-5cba-46fb-a4ce-f29a4be053b1", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f1c9801c9d0 con 0x7f1ca8073f40 2024-09-10T22:34:39.957 INFO:teuthology.orchestra.run.smithi094.stdout:0 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8073f40 msgr2=0x7f1ca814f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8073f40 0x7f1ca814f4b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800c3c0 tx=0x7f1c9800c880 comp rx=0 tx=0).stop 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 shutdown_connections 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ca8151960 0x7f1ca814f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1ca8073f40 0x7f1ca814f4b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 --2- 172.21.15.94:0/304372824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ca806f690 0x7f1ca81f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 >> 172.21.15.94:0/304372824 conn(0x7f1ca80b16d0 msgr2=0x7f1ca8072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:39.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 shutdown_connections 2024-09-10T22:34:39.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:39.956+0000 7f1caf233640 1 -- 172.21.15.94:0/304372824 wait complete. 2024-09-10T22:34:39.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 35dec180-b760-4a24-93ab-3b1573a03614 1 2024-09-10T22:34:40.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.069+0000 7f50fcac5640 1 -- 172.21.15.94:0/854556976 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 msgr2=0x7f50f815f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.069+0000 7f50fcac5640 1 --2- 172.21.15.94:0/854556976 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 0x7f50f815f2b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009860 tx=0x7f50e802a5a0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.069+0000 7f50fcac5640 1 -- 172.21.15.94:0/854556976 shutdown_connections 2024-09-10T22:34:40.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.069+0000 7f50fcac5640 1 --2- 172.21.15.94:0/854556976 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 0x7f50f815f2b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.069+0000 7f50fcac5640 1 -- 172.21.15.94:0/854556976 >> 172.21.15.94:0/854556976 conn(0x7f50f80d0c30 msgr2=0x7f50f80d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.069+0000 7f50fcac5640 1 -- 172.21.15.94:0/854556976 shutdown_connections 2024-09-10T22:34:40.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.069+0000 7f50fcac5640 1 -- 172.21.15.94:0/854556976 wait complete. 2024-09-10T22:34:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.070+0000 7f50fcac5640 1 Processor -- start 2024-09-10T22:34:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.070+0000 7f50fcac5640 1 -- start start 2024-09-10T22:34:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50fcac5640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50f8073690 0x7f50f80b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50fcac5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f80cb8b0 0x7f50f80c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50fcac5640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 0x7f50f814ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50fcac5640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80cc590 con 0x7f50f8073690 2024-09-10T22:34:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50fcac5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50f80cc410 con 0x7f50f80cb8b0 2024-09-10T22:34:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50fcac5640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f50f80cc710 con 0x7f50f815ad20 2024-09-10T22:34:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f80cb8b0 0x7f50f80c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50f6d9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 0x7f50f814ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f80cb8b0 0x7f50f80c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46626/0 (socket says 172.21.15.94:46626) 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50f6d9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 0x7f50f814ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:41560/0 (socket says 172.21.15.94:41560) 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 -- 172.21.15.94:0/2924065475 learned_addr learned my addr 172.21.15.94:0/2924065475 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50f659c640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50f8073690 0x7f50f80b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 -- 172.21.15.94:0/2924065475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 msgr2=0x7f50f814ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 0x7f50f814ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 -- 172.21.15.94:0/2924065475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50f8073690 msgr2=0x7f50f80b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50f8073690 0x7f50f80b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 -- 172.21.15.94:0/2924065475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f80ca580 con 0x7f50f80cb8b0 2024-09-10T22:34:40.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.071+0000 7f50effff640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f80cb8b0 0x7f50f80c9ac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c500 tx=0x7f50e000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.072+0000 7f50eeffd640 1 -- 172.21.15.94:0/2924065475 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0017070 con 0x7f50f80cb8b0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.072+0000 7f50eeffd640 1 -- 172.21.15.94:0/2924065475 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50e00035f0 con 0x7f50f80cb8b0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.072+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f81f6300 con 0x7f50f80cb8b0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.072+0000 7f50eeffd640 1 -- 172.21.15.94:0/2924065475 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003a40 con 0x7f50f80cb8b0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.072+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81558c0 con 0x7f50f80cb8b0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.072+0000 7f50eeffd640 1 -- 172.21.15.94:0/2924065475 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f50e0003be0 con 0x7f50f80cb8b0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.072+0000 7f50eeffd640 1 -- 172.21.15.94:0/2924065475 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f50e0012a00 con 0x7f50f80cb8b0 2024-09-10T22:34:40.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.073+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f80ce5b0 con 0x7f50f80cb8b0 2024-09-10T22:34:40.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.075+0000 7f50eeffd640 1 -- 172.21.15.94:0/2924065475 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f50f80ce5b0 con 0x7f50f80cb8b0 2024-09-10T22:34:40.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.166+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "35dec180-b760-4a24-93ab-3b1573a03614", "id": 1} v 0) -- 0x7f50f806fbe0 con 0x7f50f80cb8b0 2024-09-10T22:34:40.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.170+0000 7f50eeffd640 1 -- 172.21.15.94:0/2924065475 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35dec180-b760-4a24-93ab-3b1573a03614", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f50f806fbe0 con 0x7f50f80cb8b0 2024-09-10T22:34:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f80cb8b0 msgr2=0x7f50f80c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f80cb8b0 0x7f50f80c9ac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c500 tx=0x7f50e000c9c0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 shutdown_connections 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50f815ad20 0x7f50f814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50f80cb8b0 0x7f50f80c9ac0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 --2- 172.21.15.94:0/2924065475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50f8073690 0x7f50f80b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 >> 172.21.15.94:0/2924065475 conn(0x7f50f80d0c30 msgr2=0x7f50f80cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 shutdown_connections 2024-09-10T22:34:40.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.171+0000 7f50fcac5640 1 -- 172.21.15.94:0/2924065475 wait complete. 2024-09-10T22:34:40.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 28d55995-27ac-4899-9a76-11ef96760d95 2 2024-09-10T22:34:40.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 -- 172.21.15.94:0/3747270156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 msgr2=0x7f297415e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 --2- 172.21.15.94:0/3747270156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297415e950 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f29741559d0 tx=0x7f296c02aa30 comp rx=0 tx=0).stop 2024-09-10T22:34:40.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 -- 172.21.15.94:0/3747270156 shutdown_connections 2024-09-10T22:34:40.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 --2- 172.21.15.94:0/3747270156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297415e950 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 --2- 172.21.15.94:0/3747270156 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2974073bb0 0x7f29740b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 --2- 172.21.15.94:0/3747270156 >> v2:172.21.15.66:3300/0 conn(0x7f2974151f80 0x7f2974073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 -- 172.21.15.94:0/3747270156 >> 172.21.15.94:0/3747270156 conn(0x7f29740b27c0 msgr2=0x7f29740b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 -- 172.21.15.94:0/3747270156 shutdown_connections 2024-09-10T22:34:40.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.283+0000 7f2979c52640 1 -- 172.21.15.94:0/3747270156 wait complete. 2024-09-10T22:34:40.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.284+0000 7f2979c52640 1 Processor -- start 2024-09-10T22:34:40.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2979c52640 1 -- start start 2024-09-10T22:34:40.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2979c52640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2974073bb0 0x7f29741f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2979c52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297414edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2979c52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2974151f80 0x7f2974152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2979c52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f297415f220 con 0x7f2974151f80 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2979c52640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f297415f0a0 con 0x7f29740b4150 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2979c52640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f297415f3a0 con 0x7f2974073bb0 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f29737fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2974073bb0 0x7f29741f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297414edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297414edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46634/0 (socket says 172.21.15.94:46634) 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2972ffd640 1 -- 172.21.15.94:0/616274494 learned_addr learned my addr 172.21.15.94:0/616274494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.285+0000 7f2973fff640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2974151f80 0x7f2974152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2972ffd640 1 -- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2974073bb0 msgr2=0x7f29741f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2972ffd640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2974073bb0 0x7f29741f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2972ffd640 1 -- 172.21.15.94:0/616274494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2974151f80 msgr2=0x7f2974152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2972ffd640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2974151f80 0x7f2974152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2972ffd640 1 -- 172.21.15.94:0/616274494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29741f5ab0 con 0x7f29740b4150 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f29737fe640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2974073bb0 0x7f29741f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2972ffd640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297414edc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f296400ea90 tx=0x7f296400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:40.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f296bfff640 1 -- 172.21.15.94:0/616274494 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296400bc90 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f296bfff640 1 -- 172.21.15.94:0/616274494 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f296400be30 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741f57d0 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f296bfff640 1 -- 172.21.15.94:0/616274494 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964004280 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741f19c0 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f296bfff640 1 -- 172.21.15.94:0/616274494 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f29640035f0 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.286+0000 7f296bfff640 1 -- 172.21.15.94:0/616274494 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2964004420 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.287+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297415e950 con 0x7f29740b4150 2024-09-10T22:34:40.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.289+0000 7f296bfff640 1 -- 172.21.15.94:0/616274494 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2964003910 con 0x7f29740b4150 2024-09-10T22:34:40.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.358+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "28d55995-27ac-4899-9a76-11ef96760d95", "id": 2} v 0) -- 0x7f29740d1540 con 0x7f29740b4150 2024-09-10T22:34:40.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.362+0000 7f296bfff640 1 -- 172.21.15.94:0/616274494 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "28d55995-27ac-4899-9a76-11ef96760d95", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2964003bd0 con 0x7f29740b4150 2024-09-10T22:34:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:2 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 msgr2=0x7f297414edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297414edc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f296400ea90 tx=0x7f296400ef50 comp rx=0 tx=0).stop 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 shutdown_connections 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2974151f80 0x7f2974152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29740b4150 0x7f297414edc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 --2- 172.21.15.94:0/616274494 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2974073bb0 0x7f29741f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 >> 172.21.15.94:0/616274494 conn(0x7f29740b27c0 msgr2=0x7f29740b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 shutdown_connections 2024-09-10T22:34:40.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.363+0000 7f2979c52640 1 -- 172.21.15.94:0/616274494 wait complete. 2024-09-10T22:34:40.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 7cb4f0ea-0296-4f5e-959e-33abec73e116 3 2024-09-10T22:34:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 -- 172.21.15.94:0/2190556638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152830 msgr2=0x7f60b00cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 --2- 172.21.15.94:0/2190556638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152830 0x7f60b00cc930 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005fd0 tx=0x7f60a002a540 comp rx=0 tx=0).stop 2024-09-10T22:34:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 -- 172.21.15.94:0/2190556638 shutdown_connections 2024-09-10T22:34:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 --2- 172.21.15.94:0/2190556638 >> v2:172.21.15.94:3300/0 conn(0x7f60b0152c10 0x7f60b0070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 --2- 172.21.15.94:0/2190556638 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152830 0x7f60b00cc930 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 -- 172.21.15.94:0/2190556638 >> 172.21.15.94:0/2190556638 conn(0x7f60b00cd6e0 msgr2=0x7f60b00cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 -- 172.21.15.94:0/2190556638 shutdown_connections 2024-09-10T22:34:40.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.473+0000 7f60b4c47640 1 -- 172.21.15.94:0/2190556638 wait complete. 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.474+0000 7f60b4c47640 1 Processor -- start 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60b4c47640 1 -- start start 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60b4c47640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b0152830 0x7f60b00c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60b4c47640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152c10 0x7f60b01f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60b4c47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b01600b0 0x7f60b00ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60b4c47640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0074870 con 0x7f60b0152830 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60b4c47640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60b00746f0 con 0x7f60b01600b0 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60b4c47640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f60b006cb20 con 0x7f60b0152c10 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152c10 0x7f60b01f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b01600b0 0x7f60b00ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60a7fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152c10 0x7f60b01f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:41588/0 (socket says 172.21.15.94:41588) 2024-09-10T22:34:40.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60a7fff640 1 -- 172.21.15.94:0/1570343398 learned_addr learned my addr 172.21.15.94:0/1570343398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.475+0000 7f60ae59c640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b0152830 0x7f60b00c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a7fff640 1 -- 172.21.15.94:0/1570343398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b01600b0 msgr2=0x7f60b00ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a7fff640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b01600b0 0x7f60b00ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a7fff640 1 -- 172.21.15.94:0/1570343398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b0152830 msgr2=0x7f60b00c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a7fff640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b0152830 0x7f60b00c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a7fff640 1 -- 172.21.15.94:0/1570343398 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b0153620 con 0x7f60b0152c10 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60ae59c640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b0152830 0x7f60b00c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a7fff640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152c10 0x7f60b01f29b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f609800e9c0 tx=0x7f609800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a6ffd640 1 -- 172.21.15.94:0/1570343398 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609800cd80 con 0x7f60b0152c10 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a6ffd640 1 -- 172.21.15.94:0/1570343398 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6098004020 con 0x7f60b0152c10 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b014f7a0 con 0x7f60b0152c10 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a6ffd640 1 -- 172.21.15.94:0/1570343398 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098004450 con 0x7f60b0152c10 2024-09-10T22:34:40.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b014fd60 con 0x7f60b0152c10 2024-09-10T22:34:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a6ffd640 1 -- 172.21.15.94:0/1570343398 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6098003980 con 0x7f60b0152c10 2024-09-10T22:34:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.476+0000 7f60a6ffd640 1 -- 172.21.15.94:0/1570343398 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f6098021d00 con 0x7f60b0152c10 2024-09-10T22:34:40.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.477+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b0076550 con 0x7f60b0152c10 2024-09-10T22:34:40.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.480+0000 7f60a6ffd640 1 -- 172.21.15.94:0/1570343398 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6098026020 con 0x7f60b0152c10 2024-09-10T22:34:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.550+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7cb4f0ea-0296-4f5e-959e-33abec73e116", "id": 3} v 0) -- 0x7f60b00cc930 con 0x7f60b0152c10 2024-09-10T22:34:40.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.553+0000 7f60a6ffd640 1 -- 172.21.15.94:0/1570343398 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7cb4f0ea-0296-4f5e-959e-33abec73e116", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f6098021440 con 0x7f60b0152c10 2024-09-10T22:34:40.555 INFO:teuthology.orchestra.run.smithi094.stdout:3 2024-09-10T22:34:40.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152c10 msgr2=0x7f60b01f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152c10 0x7f60b01f29b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f609800e9c0 tx=0x7f609800ee80 comp rx=0 tx=0).stop 2024-09-10T22:34:40.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 shutdown_connections 2024-09-10T22:34:40.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b01600b0 0x7f60b00ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f60b0152c10 0x7f60b01f29b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 --2- 172.21.15.94:0/1570343398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60b0152830 0x7f60b00c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 >> 172.21.15.94:0/1570343398 conn(0x7f60b00cd6e0 msgr2=0x7f60b015c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 shutdown_connections 2024-09-10T22:34:40.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.555+0000 7f60b4c47640 1 -- 172.21.15.94:0/1570343398 wait complete. 2024-09-10T22:34:40.567 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new a9663308-591b-44e9-856b-c0666fd661b5 4 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 -- 172.21.15.94:0/2750181971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed0152c10 msgr2=0x7feed0070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 --2- 172.21.15.94:0/2750181971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed0152c10 0x7feed0070f30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feed00cd6b0 tx=0x7feec002a540 comp rx=0 tx=0).stop 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 -- 172.21.15.94:0/2750181971 shutdown_connections 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 --2- 172.21.15.94:0/2750181971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed0152c10 0x7feed0070f30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 --2- 172.21.15.94:0/2750181971 >> v2:172.21.15.66:3300/0 conn(0x7feed0152830 0x7feed00cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 -- 172.21.15.94:0/2750181971 >> 172.21.15.94:0/2750181971 conn(0x7feed00cd6e0 msgr2=0x7feed00cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 -- 172.21.15.94:0/2750181971 shutdown_connections 2024-09-10T22:34:40.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.667+0000 7feed827f640 1 -- 172.21.15.94:0/2750181971 wait complete. 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.668+0000 7feed827f640 1 Processor -- start 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.668+0000 7feed827f640 1 -- start start 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed827f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed0152830 0x7feed01576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed827f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feed0152c10 0x7feed01f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed827f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed01600b0 0x7feed00ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed827f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feed0074870 con 0x7feed0152830 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed827f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feed00746f0 con 0x7feed01600b0 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed827f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feed006cb20 con 0x7feed0152c10 2024-09-10T22:34:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed581a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feed0152c10 0x7feed01f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed581a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feed0152c10 0x7feed01f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:41592/0 (socket says 172.21.15.94:41592) 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed681c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed01600b0 0x7feed00ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed581a640 1 -- 172.21.15.94:0/4253666864 learned_addr learned my addr 172.21.15.94:0/4253666864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed601b640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed0152830 0x7feed01576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed681c640 1 -- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feed0152c10 msgr2=0x7feed01f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed681c640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feed0152c10 0x7feed01f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed681c640 1 -- 172.21.15.94:0/4253666864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed0152830 msgr2=0x7feed01576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed681c640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed0152830 0x7feed01576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed681c640 1 -- 172.21.15.94:0/4253666864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed014a240 con 0x7feed01600b0 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed581a640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feed0152c10 0x7feed01f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.669+0000 7feed681c640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed01600b0 0x7feed00ce830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feec800c730 tx=0x7feec800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:40.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.670+0000 7feec6ffd640 1 -- 172.21.15.94:0/4253666864 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8003b70 con 0x7feed01600b0 2024-09-10T22:34:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.670+0000 7feec6ffd640 1 -- 172.21.15.94:0/4253666864 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feec8003d10 con 0x7feed01600b0 2024-09-10T22:34:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.670+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed00c76d0 con 0x7feed01600b0 2024-09-10T22:34:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.670+0000 7feec6ffd640 1 -- 172.21.15.94:0/4253666864 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8005330 con 0x7feed01600b0 2024-09-10T22:34:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.670+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed014a4e0 con 0x7feed01600b0 2024-09-10T22:34:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.670+0000 7feec6ffd640 1 -- 172.21.15.94:0/4253666864 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7feec800a640 con 0x7feed01600b0 2024-09-10T22:34:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.670+0000 7feec6ffd640 1 -- 172.21.15.94:0/4253666864 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7feec8012bb0 con 0x7feed01600b0 2024-09-10T22:34:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.671+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed0076550 con 0x7feed01600b0 2024-09-10T22:34:40.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.674+0000 7feec6ffd640 1 -- 172.21.15.94:0/4253666864 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7feec80056e0 con 0x7feed01600b0 2024-09-10T22:34:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.745+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a9663308-591b-44e9-856b-c0666fd661b5", "id": 4} v 0) -- 0x7feed015dec0 con 0x7feed01600b0 2024-09-10T22:34:40.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feec6ffd640 1 -- 172.21.15.94:0/4253666864 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9663308-591b-44e9-856b-c0666fd661b5", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7feec800ab40 con 0x7feed01600b0 2024-09-10T22:34:40.750 INFO:teuthology.orchestra.run.smithi094.stdout:4 2024-09-10T22:34:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed01600b0 msgr2=0x7feed00ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feed827f640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed01600b0 0x7feed00ce830 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feec800c730 tx=0x7feec800cbf0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 shutdown_connections 2024-09-10T22:34:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feed827f640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feed01600b0 0x7feed00ce830 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feed827f640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feed0152c10 0x7feed01f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feed827f640 1 --2- 172.21.15.94:0/4253666864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feed0152830 0x7feed01576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.749+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 >> 172.21.15.94:0/4253666864 conn(0x7feed00cd6e0 msgr2=0x7feed015d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.750+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 shutdown_connections 2024-09-10T22:34:40.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.750+0000 7feed827f640 1 -- 172.21.15.94:0/4253666864 wait complete. 2024-09-10T22:34:40.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new ad4f4b42-77cc-431d-8996-d6f4872cb660 5 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 -- 172.21.15.94:0/3310538405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 msgr2=0x7f36fc141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 --2- 172.21.15.94:0/3310538405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc141810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005fd0 tx=0x7f36f802a5d0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 -- 172.21.15.94:0/3310538405 shutdown_connections 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 --2- 172.21.15.94:0/3310538405 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc141810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 --2- 172.21.15.94:0/3310538405 >> v2:172.21.15.66:3300/0 conn(0x7f36fc13ed30 0x7f36fc06cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 -- 172.21.15.94:0/3310538405 >> 172.21.15.94:0/3310538405 conn(0x7f36fc0b27c0 msgr2=0x7f36fc0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 -- 172.21.15.94:0/3310538405 shutdown_connections 2024-09-10T22:34:40.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.862+0000 7f3703552640 1 -- 172.21.15.94:0/3310538405 wait complete. 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.863+0000 7f3703552640 1 Processor -- start 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.863+0000 7f3703552640 1 -- start start 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3703552640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3703552640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36fc13ed30 0x7f36fc0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3703552640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36fc141d50 0x7f36fc1f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3703552640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc0d05f0 con 0x7f36fc141d50 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3703552640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc0d0470 con 0x7f36fc13ed30 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3703552640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc160160 con 0x7f36fc06d060 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3700aed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36fc13ed30 0x7f36fc0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:41602/0 (socket says 172.21.15.94:41602) 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 -- 172.21.15.94:0/1225315289 learned_addr learned my addr 172.21.15.94:0/1225315289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3701aef640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36fc141d50 0x7f36fc1f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 -- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36fc13ed30 msgr2=0x7f36fc0710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36fc13ed30 0x7f36fc0710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 -- 172.21.15.94:0/1225315289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36fc141d50 msgr2=0x7f36fc1f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36fc141d50 0x7f36fc1f1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f37012ee640 1 -- 172.21.15.94:0/1225315289 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc1f5ed0 con 0x7f36fc06d060 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.864+0000 7f3700aed640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36fc13ed30 0x7f36fc0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f37012ee640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc156b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f36edffb640 1 -- 172.21.15.94:0/1225315289 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0017070 con 0x7f36fc06d060 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f36edffb640 1 -- 172.21.15.94:0/1225315289 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f0009d40 con 0x7f36fc06d060 2024-09-10T22:34:40.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36fc1f61b0 con 0x7f36fc06d060 2024-09-10T22:34:40.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f36edffb640 1 -- 172.21.15.94:0/1225315289 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0004280 con 0x7f36fc06d060 2024-09-10T22:34:40.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f36fc157060 con 0x7f36fc06d060 2024-09-10T22:34:40.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f36edffb640 1 -- 172.21.15.94:0/1225315289 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f36f00035f0 con 0x7f36fc06d060 2024-09-10T22:34:40.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.865+0000 7f36edffb640 1 -- 172.21.15.94:0/1225315289 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f36f0010c60 con 0x7f36fc06d060 2024-09-10T22:34:40.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.866+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36fc160610 con 0x7f36fc06d060 2024-09-10T22:34:40.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.868+0000 7f36edffb640 1 -- 172.21.15.94:0/1225315289 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f36f0004420 con 0x7f36fc06d060 2024-09-10T22:34:40.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.939+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ad4f4b42-77cc-431d-8996-d6f4872cb660", "id": 5} v 0) -- 0x7f36fc069840 con 0x7f36fc06d060 2024-09-10T22:34:40.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.943+0000 7f36edffb640 1 -- 172.21.15.94:0/1225315289 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad4f4b42-77cc-431d-8996-d6f4872cb660", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f36f0003af0 con 0x7f36fc06d060 2024-09-10T22:34:40.944 INFO:teuthology.orchestra.run.smithi094.stdout:5 2024-09-10T22:34:40.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 msgr2=0x7f36fc156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:40.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc156b20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d950 tx=0x7f36f000de10 comp rx=0 tx=0).stop 2024-09-10T22:34:40.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 shutdown_connections 2024-09-10T22:34:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36fc141d50 0x7f36fc1f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36fc13ed30 0x7f36fc0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 --2- 172.21.15.94:0/1225315289 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f36fc06d060 0x7f36fc156b20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 >> 172.21.15.94:0/1225315289 conn(0x7f36fc0b27c0 msgr2=0x7f36fc06ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 shutdown_connections 2024-09-10T22:34:40.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:40.944+0000 7f3703552640 1 -- 172.21.15.94:0/1225315289 wait complete. 2024-09-10T22:34:40.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new 9527f394-d0d3-4abe-a5f5-91dde0438935 6 2024-09-10T22:34:41.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.055+0000 7f5cdfb10640 1 -- 172.21.15.94:0/4027585846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 msgr2=0x7f5cd80b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.055+0000 7f5cdfb10640 1 --2- 172.21.15.94:0/4027585846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd80b3c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009860 tx=0x7f5cc802a5a0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.055+0000 7f5cdfb10640 1 -- 172.21.15.94:0/4027585846 shutdown_connections 2024-09-10T22:34:41.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.055+0000 7f5cdfb10640 1 --2- 172.21.15.94:0/4027585846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd80b3c10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.055+0000 7f5cdfb10640 1 -- 172.21.15.94:0/4027585846 >> 172.21.15.94:0/4027585846 conn(0x7f5cd80b27c0 msgr2=0x7f5cd80b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:41.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.055+0000 7f5cdfb10640 1 -- 172.21.15.94:0/4027585846 shutdown_connections 2024-09-10T22:34:41.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.055+0000 7f5cdfb10640 1 -- 172.21.15.94:0/4027585846 wait complete. 2024-09-10T22:34:41.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.057+0000 7f5cdfb10640 1 Processor -- start 2024-09-10T22:34:41.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.057+0000 7f5cdfb10640 1 -- start start 2024-09-10T22:34:41.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdfb10640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd8150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdfb10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cd80b4150 0x7f5cd81f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdfb10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd8151f80 0x7f5cd81f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdfb10640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd815dd30 con 0x7f5cd80b4150 2024-09-10T22:34:41.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdfb10640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd815dbb0 con 0x7f5cd8151f80 2024-09-10T22:34:41.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdfb10640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd815deb0 con 0x7f5cd8073d80 2024-09-10T22:34:41.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdd8ac640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd8150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cde0ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd8151f80 0x7f5cd81f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdd8ac640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd8150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:41616/0 (socket says 172.21.15.94:41616) 2024-09-10T22:34:41.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cde0ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd8151f80 0x7f5cd81f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46706/0 (socket says 172.21.15.94:46706) 2024-09-10T22:34:41.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdd8ac640 1 -- 172.21.15.94:0/3223170334 learned_addr learned my addr 172.21.15.94:0/3223170334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:41.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdd0ab640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cd80b4150 0x7f5cd81f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cde0ad640 1 -- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 msgr2=0x7f5cd8150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cde0ad640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd8150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cde0ad640 1 -- 172.21.15.94:0/3223170334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cd80b4150 msgr2=0x7f5cd81f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cde0ad640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cd80b4150 0x7f5cd81f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cde0ad640 1 -- 172.21.15.94:0/3223170334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd81f3bc0 con 0x7f5cd8151f80 2024-09-10T22:34:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.058+0000 7f5cdd8ac640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd8150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cde0ad640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd8151f80 0x7f5cd81f6940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c730 tx=0x7f5cd000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cce7fc640 1 -- 172.21.15.94:0/3223170334 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0003b70 con 0x7f5cd8151f80 2024-09-10T22:34:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cce7fc640 1 -- 172.21.15.94:0/3223170334 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd0003d10 con 0x7f5cd8151f80 2024-09-10T22:34:41.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd81f3600 con 0x7f5cd8151f80 2024-09-10T22:34:41.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cce7fc640 1 -- 172.21.15.94:0/3223170334 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000aab0 con 0x7f5cd8151f80 2024-09-10T22:34:41.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd814fa90 con 0x7f5cd8151f80 2024-09-10T22:34:41.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cce7fc640 1 -- 172.21.15.94:0/3223170334 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5cd000ac50 con 0x7f5cd8151f80 2024-09-10T22:34:41.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.059+0000 7f5cce7fc640 1 -- 172.21.15.94:0/3223170334 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f5cd00056c0 con 0x7f5cd8151f80 2024-09-10T22:34:41.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.060+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd806a050 con 0x7f5cd8151f80 2024-09-10T22:34:41.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.063+0000 7f5cce7fc640 1 -- 172.21.15.94:0/3223170334 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5cd806a050 con 0x7f5cd8151f80 2024-09-10T22:34:41.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.134+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9527f394-d0d3-4abe-a5f5-91dde0438935", "id": 6} v 0) -- 0x7f5cd80b3c10 con 0x7f5cd8151f80 2024-09-10T22:34:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.138+0000 7f5cce7fc640 1 -- 172.21.15.94:0/3223170334 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9527f394-d0d3-4abe-a5f5-91dde0438935", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f5cd00050d0 con 0x7f5cd8151f80 2024-09-10T22:34:41.139 INFO:teuthology.orchestra.run.smithi094.stdout:6 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd8151f80 msgr2=0x7f5cd81f6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd8151f80 0x7f5cd81f6940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c730 tx=0x7f5cd000cbf0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 shutdown_connections 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cd8151f80 0x7f5cd81f6940 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5cd80b4150 0x7f5cd81f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 --2- 172.21.15.94:0/3223170334 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5cd8073d80 0x7f5cd8150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 >> 172.21.15.94:0/3223170334 conn(0x7f5cd80b27c0 msgr2=0x7f5cd8070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 shutdown_connections 2024-09-10T22:34:41.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.139+0000 7f5cdfb10640 1 -- 172.21.15.94:0/3223170334 wait complete. 2024-09-10T22:34:41.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd new f372de29-523e-443d-92db-1012f33193c1 7 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 -- 172.21.15.94:0/1835166796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 msgr2=0x7f1b1c0b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 --2- 172.21.15.94:0/1835166796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c0b2b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009820 tx=0x7f1b0802e920 comp rx=0 tx=0).stop 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 -- 172.21.15.94:0/1835166796 shutdown_connections 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 --2- 172.21.15.94:0/1835166796 >> v2:172.21.15.94:3300/0 conn(0x7f1b1c15e670 0x7f1b1c15ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 --2- 172.21.15.94:0/1835166796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c0b2b50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 -- 172.21.15.94:0/1835166796 >> 172.21.15.94:0/1835166796 conn(0x7f1b1c0cd6e0 msgr2=0x7f1b1c0cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 -- 172.21.15.94:0/1835166796 shutdown_connections 2024-09-10T22:34:41.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.250+0000 7f1b22e47640 1 -- 172.21.15.94:0/1835166796 wait complete. 2024-09-10T22:34:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.251+0000 7f1b22e47640 1 Processor -- start 2024-09-10T22:34:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.251+0000 7f1b22e47640 1 -- start start 2024-09-10T22:34:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b22e47640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b22e47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c15ab20 0x7f1b1c1517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b22e47640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b1c15e670 0x7f1b1c0b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b22e47640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c06f9b0 con 0x7f1b1c15e670 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b22e47640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c06f830 con 0x7f1b1c15ab20 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b22e47640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1c06fb30 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c15ab20 0x7f1b1c1517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.94:41636/0 (socket says 172.21.15.94:41636) 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 -- 172.21.15.94:0/1277177907 learned_addr learned my addr 172.21.15.94:0/1277177907 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b213e4640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b1c15e670 0x7f1b1c0b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 -- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c15ab20 msgr2=0x7f1b1c1517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c15ab20 0x7f1b1c1517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 -- 172.21.15.94:0/1277177907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b1c15e670 msgr2=0x7f1b1c0b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b1c15e670 0x7f1b1c0b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b20be3640 1 -- 172.21.15.94:0/1277177907 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c152740 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.252+0000 7f1b1bfff640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c15ab20 0x7f1b1c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b20be3640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c155860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c760 tx=0x7f1b1000cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b197fa640 1 -- 172.21.15.94:0/1277177907 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10016020 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b197fa640 1 -- 172.21.15.94:0/1277177907 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b1000a930 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b197fa640 1 -- 172.21.15.94:0/1277177907 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000ad80 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c0c4790 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c0c4d50 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b197fa640 1 -- 172.21.15.94:0/1277177907 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1b100127b0 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.253+0000 7f1b197fa640 1 -- 172.21.15.94:0/1277177907 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f1b1001bae0 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.254+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c076550 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.256+0000 7f1b197fa640 1 -- 172.21.15.94:0/1277177907 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1b10020020 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.327+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f372de29-523e-443d-92db-1012f33193c1", "id": 7} v 0) -- 0x7f1b1c070300 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.330+0000 7f1b197fa640 1 -- 172.21.15.94:0/1277177907 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f372de29-523e-443d-92db-1012f33193c1", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f1b1c0cb800 con 0x7f1b1c0b26e0 2024-09-10T22:34:41.332 INFO:teuthology.orchestra.run.smithi094.stdout:7 2024-09-10T22:34:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 msgr2=0x7f1b1c155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c155860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c760 tx=0x7f1b1000cc20 comp rx=0 tx=0).stop 2024-09-10T22:34:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 shutdown_connections 2024-09-10T22:34:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b1c15e670 0x7f1b1c0b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b1c15ab20 0x7f1b1c1517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 --2- 172.21.15.94:0/1277177907 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1b1c0b26e0 0x7f1b1c155860 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 >> 172.21.15.94:0/1277177907 conn(0x7f1b1c0cd6e0 msgr2=0x7f1b1c0ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 shutdown_connections 2024-09-10T22:34:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:34:41.331+0000 7f1b22e47640 1 -- 172.21.15.94:0/1277177907 wait complete. 2024-09-10T22:34:41.343 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T22:34:41.343 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T22:34:41.345 INFO:tasks.ceph.osd.0:Started 2024-09-10T22:34:41.346 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T22:34:41.346 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T22:34:41.348 INFO:tasks.ceph.osd.1:Started 2024-09-10T22:34:41.348 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T22:34:41.348 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T22:34:41.350 INFO:tasks.ceph.osd.2:Started 2024-09-10T22:34:41.351 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T22:34:41.351 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T22:34:41.353 INFO:tasks.ceph.osd.3:Started 2024-09-10T22:34:41.353 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T22:34:41.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T22:34:41.356 INFO:tasks.ceph.osd.4:Started 2024-09-10T22:34:41.356 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T22:34:41.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T22:34:41.359 INFO:tasks.ceph.osd.5:Started 2024-09-10T22:34:41.359 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T22:34:41.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T22:34:41.361 INFO:tasks.ceph.osd.6:Started 2024-09-10T22:34:41.361 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T22:34:41.362 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T22:34:41.364 INFO:tasks.ceph.osd.7:Started 2024-09-10T22:34:41.364 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:34:41.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.514+0000 7fbb48b94640 1 -- 172.21.15.66:0/1568297445 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4415bf00 msgr2=0x7fbb440d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.514+0000 7fbb48b94640 1 --2- 172.21.15.66:0/1568297445 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4415bf00 0x7fbb440d1cb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009850 tx=0x7fbb3402a4f0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.514+0000 7fbb48b94640 1 -- 172.21.15.66:0/1568297445 shutdown_connections 2024-09-10T22:34:41.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.514+0000 7fbb48b94640 1 --2- 172.21.15.66:0/1568297445 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb4415bf00 0x7fbb440d1cb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.514+0000 7fbb48b94640 1 -- 172.21.15.66:0/1568297445 >> 172.21.15.66:0/1568297445 conn(0x7fbb4409aa70 msgr2=0x7fbb4409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:41.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.514+0000 7fbb48b94640 1 -- 172.21.15.66:0/1568297445 shutdown_connections 2024-09-10T22:34:41.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.514+0000 7fbb48b94640 1 -- 172.21.15.66:0/1568297445 wait complete. 2024-09-10T22:34:41.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.515+0000 7fbb48b94640 1 Processor -- start 2024-09-10T22:34:41.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb48b94640 1 -- start start 2024-09-10T22:34:41.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb48b94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb440b2d40 0x7fbb441f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb48b94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb440b3380 0x7fbb441f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb48b94640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4415bf00 0x7fbb441f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb48b94640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb440d2600 con 0x7fbb4415bf00 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb48b94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb440d2480 con 0x7fbb440b3380 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb48b94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbb440b0c50 con 0x7fbb440b2d40 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb42d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4415bf00 0x7fbb441f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb42d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4415bf00 0x7fbb441f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57308/0 (socket says 172.21.15.66:57308) 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb42d9d640 1 -- 172.21.15.66:0/1965406796 learned_addr learned my addr 172.21.15.66:0/1965406796 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.516+0000 7fbb41d9b640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb440b3380 0x7fbb441f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb4259c640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb440b2d40 0x7fbb441f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb41d9b640 1 -- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb440b2d40 msgr2=0x7fbb441f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb41d9b640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb440b2d40 0x7fbb441f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb41d9b640 1 -- 172.21.15.66:0/1965406796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4415bf00 msgr2=0x7fbb441f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb41d9b640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4415bf00 0x7fbb441f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb41d9b640 1 -- 172.21.15.66:0/1965406796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb441f5710 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb42d9d640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4415bf00 0x7fbb441f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb41d9b640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb440b3380 0x7fbb441f75d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800c400 tx=0x7fbb3800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb32ffd640 1 -- 172.21.15.66:0/1965406796 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38011790 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb32ffd640 1 -- 172.21.15.66:0/1965406796 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb38017070 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb32ffd640 1 -- 172.21.15.66:0/1965406796 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38011d40 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4406e1f0 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4406e7b0 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb32ffd640 1 -- 172.21.15.66:0/1965406796 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fbb3801c460 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.517+0000 7fbb32ffd640 1 -- 172.21.15.66:0/1965406796 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fbb38012790 con 0x7fbb440b3380 2024-09-10T22:34:41.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.518+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb440d1cb0 con 0x7fbb440b3380 2024-09-10T22:34:41.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.522+0000 7fbb32ffd640 1 -- 172.21.15.66:0/1965406796 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fbb441f5710 con 0x7fbb440b3380 2024-09-10T22:34:41.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.584+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb44071160 con 0x7fbb440b3380 2024-09-10T22:34:41.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.585+0000 7fbb32ffd640 1 -- 172.21.15.66:0/1965406796 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fbb38012ab0 con 0x7fbb440b3380 2024-09-10T22:34:41.586 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:41.587 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":10,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:34:41.329138+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:34:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb440b3380 msgr2=0x7fbb441f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb440b3380 0x7fbb441f75d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800c400 tx=0x7fbb3800c8c0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 shutdown_connections 2024-09-10T22:34:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb4415bf00 0x7fbb441f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb440b3380 0x7fbb441f75d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 --2- 172.21.15.66:0/1965406796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb440b2d40 0x7fbb441f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:41.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 >> 172.21.15.66:0/1965406796 conn(0x7fbb4409aa70 msgr2=0x7fbb440ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.586+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 shutdown_connections 2024-09-10T22:34:41.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:41.587+0000 7fbb48b94640 1 -- 172.21.15.66:0/1965406796 wait complete. 2024-09-10T22:34:41.599 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T22:34:41.599 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T22:34:42.322 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:34:42.321+0000 7f3f674ed640 -1 Falling back to public interface 2024-09-10T22:34:42.405 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 -- 172.21.15.66:0/1783319261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0d2630 msgr2=0x7f831c0cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 --2- 172.21.15.66:0/1783319261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0d2630 0x7f831c0cfc40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8314005fd0 tx=0x7f831402aa90 comp rx=0 tx=0).stop 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 -- 172.21.15.66:0/1783319261 shutdown_connections 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 --2- 172.21.15.66:0/1783319261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0d2630 0x7f831c0cfc40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 --2- 172.21.15.66:0/1783319261 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f831c1618b0 0x7f831c0d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 -- 172.21.15.66:0/1783319261 >> 172.21.15.66:0/1783319261 conn(0x7f831c0d4020 msgr2=0x7f831c0d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 -- 172.21.15.66:0/1783319261 shutdown_connections 2024-09-10T22:34:42.502 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.500+0000 7f8320b82640 1 -- 172.21.15.66:0/1783319261 wait complete. 2024-09-10T22:34:42.503 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 Processor -- start 2024-09-10T22:34:42.503 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 -- start start 2024-09-10T22:34:42.505 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0b55d0 0x7f831c1f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:42.505 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0d2630 0x7f831c150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:42.505 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f831c1618b0 0x7f831c1f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0b3b00 con 0x7f831c0b55d0 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0b3980 con 0x7f831c0d2630 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.502+0000 7f8320b82640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0b3c80 con 0x7f831c1618b0 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0b55d0 0x7f831c1f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0b55d0 0x7f831c1f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57330/0 (socket says 172.21.15.66:57330) 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 -- 172.21.15.66:0/1015448401 learned_addr learned my addr 172.21.15.66:0/1015448401 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f8313fff640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0d2630 0x7f831c150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831ad9d640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f831c1618b0 0x7f831c1f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 -- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f831c1618b0 msgr2=0x7f831c1f7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f831c1618b0 0x7f831c1f7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 -- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0d2630 msgr2=0x7f831c150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:42.506 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0d2630 0x7f831c150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 -- 172.21.15.66:0/1015448401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1f8920 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f831a59c640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0b55d0 0x7f831c1f5310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f830c00d750 tx=0x7f830c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f8312ffd640 1 -- 172.21.15.66:0/1015448401 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c017070 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f8312ffd640 1 -- 172.21.15.66:0/1015448401 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f830c004310 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1f8c00 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f8312ffd640 1 -- 172.21.15.66:0/1015448401 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c00de60 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1f4b10 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.503+0000 7f8312ffd640 1 -- 172.21.15.66:0/1015448401 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f830c005020 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.504+0000 7f8312ffd640 1 -- 172.21.15.66:0/1015448401 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f830c003cc0 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.504+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c0d1360 con 0x7f831c0b55d0 2024-09-10T22:34:42.507 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.506+0000 7f8312ffd640 1 -- 172.21.15.66:0/1015448401 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f830c020020 con 0x7f831c0b55d0 2024-09-10T22:34:42.545 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:34:42.544+0000 7f33347bd640 -1 Falling back to public interface 2024-09-10T22:34:42.569 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.568+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f831c0cf7e0 con 0x7f831c0b55d0 2024-09-10T22:34:42.570 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:34:42.568+0000 7f8380e1e640 -1 Falling back to public interface 2024-09-10T22:34:42.570 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.569+0000 7f8312ffd640 1 -- 172.21.15.66:0/1015448401 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f830c005340 con 0x7f831c0b55d0 2024-09-10T22:34:42.571 INFO:teuthology.misc.health.smithi066.stdout: 2024-09-10T22:34:42.571 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":10,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:34:41.329138+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:34:42.572 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0b55d0 msgr2=0x7f831c1f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:42.572 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0b55d0 0x7f831c1f5310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f830c00d750 tx=0x7f830c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:34:42.572 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 shutdown_connections 2024-09-10T22:34:42.572 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f831c1618b0 0x7f831c1f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:42.572 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f831c0d2630 0x7f831c150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:42.573 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 --2- 172.21.15.66:0/1015448401 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f831c0b55d0 0x7f831c1f5310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:42.573 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 >> 172.21.15.66:0/1015448401 conn(0x7f831c0d4020 msgr2=0x7f831c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:42.573 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 shutdown_connections 2024-09-10T22:34:42.573 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:42.570+0000 7f8320b82640 1 -- 172.21.15.66:0/1015448401 wait complete. 2024-09-10T22:34:42.580 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T22:34:42.829 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:34:42.828+0000 7effdfc17640 -1 Falling back to public interface 2024-09-10T22:34:43.097 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:34:43.094+0000 7f765180d640 -1 Falling back to public interface 2024-09-10T22:34:43.120 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:34:43.118+0000 7f0cb6a14640 -1 Falling back to public interface 2024-09-10T22:34:43.341 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:34:43.338+0000 7fce62a0e640 -1 Falling back to public interface 2024-09-10T22:34:43.367 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:34:43.365+0000 7f806520d640 -1 Falling back to public interface 2024-09-10T22:34:43.599 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:34:43.597+0000 7f39da7de640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T22:34:43.599 INFO:tasks.ceph.mgr.x.smithi094.stderr:2024-09-10T22:34:43.597+0000 7f39da7de640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T22:34:46.135 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:34:46.134+0000 7f8380e1e640 -1 osd.2 0 log_to_monitors true 2024-09-10T22:34:46.138 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:34:46.137+0000 7f3f674ed640 -1 osd.1 0 log_to_monitors true 2024-09-10T22:34:46.369 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:34:46.368+0000 7effdfc17640 -1 osd.3 0 log_to_monitors true 2024-09-10T22:34:46.703 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:34:46.700+0000 7f0cb6a14640 -1 osd.7 0 log_to_monitors true 2024-09-10T22:34:46.801 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:34:46.800+0000 7f33347bd640 -1 osd.0 0 log_to_monitors true 2024-09-10T22:34:47.191 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:34:47.189+0000 7f765180d640 -1 osd.5 0 log_to_monitors true 2024-09-10T22:34:47.377 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:34:47.374+0000 7fce62a0e640 -1 osd.4 0 log_to_monitors true 2024-09-10T22:34:47.416 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:34:47.414+0000 7f806520d640 -1 osd.6 0 log_to_monitors true 2024-09-10T22:34:49.387 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T22:34:49.431 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:34:49.430+0000 7effdb601640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T22:34:49.443 INFO:tasks.ceph.osd.3.smithi066.stderr:2024-09-10T22:34:49.442+0000 7effd3420640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:49.495 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:34:49.494+0000 7f3f63c90640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T22:34:49.506 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:34:49.505+0000 7f837c85e640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T22:34:49.508 INFO:tasks.ceph.osd.1.smithi066.stderr:2024-09-10T22:34:49.507+0000 7f3f5b29b640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:49.517 INFO:tasks.ceph.osd.2.smithi066.stderr:2024-09-10T22:34:49.516+0000 7f837467d640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:49.539 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.537+0000 7fd39074b640 1 -- 172.21.15.66:0/4110052624 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880b55d0 msgr2=0x7fd3880cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:49.539 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.537+0000 7fd39074b640 1 --2- 172.21.15.66:0/4110052624 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880b55d0 0x7fd3880cfc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd378009860 tx=0x7fd37802a510 comp rx=0 tx=0).stop 2024-09-10T22:34:49.539 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.537+0000 7fd39074b640 1 -- 172.21.15.66:0/4110052624 shutdown_connections 2024-09-10T22:34:49.539 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.538+0000 7fd39074b640 1 --2- 172.21.15.66:0/4110052624 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880b55d0 0x7fd3880cfc40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:49.539 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.538+0000 7fd39074b640 1 -- 172.21.15.66:0/4110052624 >> 172.21.15.66:0/4110052624 conn(0x7fd3880d4020 msgr2=0x7fd3880d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:49.539 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.538+0000 7fd39074b640 1 -- 172.21.15.66:0/4110052624 shutdown_connections 2024-09-10T22:34:49.540 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.538+0000 7fd39074b640 1 -- 172.21.15.66:0/4110052624 wait complete. 2024-09-10T22:34:49.540 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.539+0000 7fd39074b640 1 Processor -- start 2024-09-10T22:34:49.540 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.539+0000 7fd39074b640 1 -- start start 2024-09-10T22:34:49.542 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd39074b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3880b55d0 0x7fd3880cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:49.542 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd39074b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880d1f20 0x7fd3881502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:49.542 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd39074b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3881618b0 0x7fd3881507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:49.542 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd39074b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880b3d10 con 0x7fd3881618b0 2024-09-10T22:34:49.542 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd39074b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880b3b90 con 0x7fd3880b55d0 2024-09-10T22:34:49.542 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd39074b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd3880b3e90 con 0x7fd3880d1f20 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd38ece8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3881618b0 0x7fd3881507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd38ece8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3881618b0 0x7fd3881507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55008/0 (socket says 172.21.15.66:55008) 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd38ece8640 1 -- 172.21.15.66:0/1616810925 learned_addr learned my addr 172.21.15.66:0/1616810925 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd38dce6640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880d1f20 0x7fd3881502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.540+0000 7fd38e4e7640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3880b55d0 0x7fd3880cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd38ece8640 1 -- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880d1f20 msgr2=0x7fd3881502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd38ece8640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880d1f20 0x7fd3881502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd38ece8640 1 -- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3880b55d0 msgr2=0x7fd3880cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd38ece8640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3880b55d0 0x7fd3880cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd38ece8640 1 -- 172.21.15.66:0/1616810925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3880b9ae0 con 0x7fd3881618b0 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd38e4e7640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3880b55d0 0x7fd3880cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd38ece8640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3881618b0 0x7fd3881507e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd38000cbc0 tx=0x7fd38000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd37effd640 1 -- 172.21.15.66:0/1616810925 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380016020 con 0x7fd3881618b0 2024-09-10T22:34:49.543 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd37effd640 1 -- 172.21.15.66:0/1616810925 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd38000ae30 con 0x7fd3881618b0 2024-09-10T22:34:49.544 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3880c8b40 con 0x7fd3881618b0 2024-09-10T22:34:49.544 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd37effd640 1 -- 172.21.15.66:0/1616810925 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380012660 con 0x7fd3881618b0 2024-09-10T22:34:49.544 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.541+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3880c9100 con 0x7fd3881618b0 2024-09-10T22:34:49.544 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.542+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3880b3320 con 0x7fd3881618b0 2024-09-10T22:34:49.545 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.543+0000 7fd37effd640 1 -- 172.21.15.66:0/1616810925 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) ==== 100545+0+0 (secure 0 0 0) 0x7fd3800128c0 con 0x7fd3881618b0 2024-09-10T22:34:49.545 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.544+0000 7fd37effd640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd36c077ee0 0x7fd36c07a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:49.545 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.544+0000 7fd37effd640 1 -- 172.21.15.66:0/1616810925 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7fd3800996e0 con 0x7fd3881618b0 2024-09-10T22:34:49.545 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.544+0000 7fd38e4e7640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd36c077ee0 0x7fd36c07a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:49.545 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.544+0000 7fd38e4e7640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd36c077ee0 0x7fd36c07a3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd384000c00 tx=0x7fd384009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:49.549 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.548+0000 7fd37effd640 1 -- 172.21.15.66:0/1616810925 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd3880b3320 con 0x7fd3881618b0 2024-09-10T22:34:49.791 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:34:49.790+0000 7f333074c640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T22:34:49.797 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.796+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd388161410 con 0x7fd3881618b0 2024-09-10T22:34:49.800 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.797+0000 7fd37effd640 1 -- 172.21.15.66:0/1616810925 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8503 (secure 0 0 0) 0x7fd388161410 con 0x7fd3881618b0 2024-09-10T22:34:49.801 INFO:teuthology.misc.health.smithi066.stdout: 2024-09-10T22:34:49.801 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":14,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:34:49.715263+0000","last_up_change":"2024-09-10T22:34:49.715263+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd36c077ee0 msgr2=0x7fd36c07a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd36c077ee0 0x7fd36c07a3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd384000c00 tx=0x7fd384009040 comp rx=0 tx=0).stop 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3881618b0 msgr2=0x7fd3881507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3881618b0 0x7fd3881507e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd38000cbc0 tx=0x7fd38000a430 comp rx=0 tx=0).stop 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 shutdown_connections 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd36c077ee0 0x7fd36c07a3a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3881618b0 0x7fd3881507e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3880d1f20 0x7fd3881502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 --2- 172.21.15.66:0/1616810925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3880b55d0 0x7fd3880cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:49.802 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 >> 172.21.15.66:0/1616810925 conn(0x7fd3880d4020 msgr2=0x7fd38806d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:49.803 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 shutdown_connections 2024-09-10T22:34:49.803 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:49.799+0000 7fd39074b640 1 -- 172.21.15.66:0/1616810925 wait complete. 2024-09-10T22:34:49.804 INFO:tasks.ceph.osd.0.smithi066.stderr:2024-09-10T22:34:49.803+0000 7f332856b640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:49.810 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-10T22:34:50.217 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:34:50.215+0000 7f0cb23e1640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T22:34:50.229 INFO:tasks.ceph.osd.7.smithi094.stderr:2024-09-10T22:34:50.226+0000 7f0caa200640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:50.244 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:34:50.242+0000 7f8060c31640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T22:34:50.258 INFO:tasks.ceph.osd.6.smithi094.stderr:2024-09-10T22:34:50.256+0000 7f8058a50640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:50.279 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:34:50.277+0000 7f764d097640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T22:34:50.293 INFO:tasks.ceph.osd.5.smithi094.stderr:2024-09-10T22:34:50.291+0000 7f7644eb6640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:50.298 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:34:50.296+0000 7fce5eb12640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T22:34:50.314 INFO:tasks.ceph.osd.4.smithi094.stderr:2024-09-10T22:34:50.312+0000 7fce5611d640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T22:34:56.615 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 -- 172.21.15.66:0/1037866913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 msgr2=0x7f5fa40cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 --2- 172.21.15.66:0/1037866913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40cfc40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c005fd0 tx=0x7f5f8c02a540 comp rx=0 tx=0).stop 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 -- 172.21.15.66:0/1037866913 shutdown_connections 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 --2- 172.21.15.66:0/1037866913 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fa40d1f20 0x7f5fa40d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 --2- 172.21.15.66:0/1037866913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40cfc40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 -- 172.21.15.66:0/1037866913 >> 172.21.15.66:0/1037866913 conn(0x7f5fa40d4020 msgr2=0x7f5fa40d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 -- 172.21.15.66:0/1037866913 shutdown_connections 2024-09-10T22:34:56.725 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.723+0000 7f5fa9495640 1 -- 172.21.15.66:0/1037866913 wait complete. 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 Processor -- start 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 -- start start 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fa40d1f20 0x7f5fa40c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41618b0 0x7f5fa4150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa40b3980 con 0x7f5fa40b55d0 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa40d4dd0 con 0x7f5fa41618b0 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.725+0000 7f5fa9495640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa40b3b00 con 0x7f5fa40d1f20 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:56.728 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36662/0 (socket says 172.21.15.66:36662) 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 -- 172.21.15.66:0/3611463790 learned_addr learned my addr 172.21.15.66:0/3611463790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa27fc640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fa40d1f20 0x7f5fa40c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa37fe640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41618b0 0x7f5fa4150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 -- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fa40d1f20 msgr2=0x7f5fa40c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fa40d1f20 0x7f5fa40c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 -- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41618b0 msgr2=0x7f5fa4150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41618b0 0x7f5fa4150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 -- 172.21.15.66:0/3611463790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa41514f0 con 0x7f5fa40b55d0 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa27fc640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fa40d1f20 0x7f5fa40c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa2ffd640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40ca8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400c730 tx=0x7f5f9400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5f9b7fe640 1 -- 172.21.15.66:0/3611463790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94003b70 con 0x7f5fa40b55d0 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5f9b7fe640 1 -- 172.21.15.66:0/3611463790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f94003d30 con 0x7f5fa40b55d0 2024-09-10T22:34:56.729 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5f9b7fe640 1 -- 172.21.15.66:0/3611463790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400aab0 con 0x7f5fa40b55d0 2024-09-10T22:34:56.730 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa40c9820 con 0x7f5fa40b55d0 2024-09-10T22:34:56.730 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.726+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa40c9de0 con 0x7f5fa40b55d0 2024-09-10T22:34:56.730 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.728+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa40d23b0 con 0x7f5fa40b55d0 2024-09-10T22:34:56.731 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.728+0000 7f5f9b7fe640 1 -- 172.21.15.66:0/3611463790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5f9400ac50 con 0x7f5fa40b55d0 2024-09-10T22:34:56.731 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.729+0000 7f5f9b7fe640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5f84077f70 0x7f5f8407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:56.731 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.729+0000 7f5f9b7fe640 1 -- 172.21.15.66:0/3611463790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5f94099800 con 0x7f5fa40b55d0 2024-09-10T22:34:56.731 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.729+0000 7f5fa27fc640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5f84077f70 0x7f5f8407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:56.731 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.729+0000 7f5fa27fc640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5f84077f70 0x7f5f8407a430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c033000 tx=0x7f5f8c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:56.734 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.733+0000 7f5f9b7fe640 1 -- 172.21.15.66:0/3611463790 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5fa41514f0 con 0x7f5fa40b55d0 2024-09-10T22:34:56.922 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.921+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fa406e7b0 con 0x7f5fa40b55d0 2024-09-10T22:34:56.923 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.922+0000 7f5f9b7fe640 1 -- 172.21.15.66:0/3611463790 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12872 (secure 0 0 0) 0x7f5f940626e0 con 0x7f5fa40b55d0 2024-09-10T22:34:56.923 INFO:teuthology.misc.health.smithi066.stdout: 2024-09-10T22:34:56.923 INFO:teuthology.misc.health.smithi066.stdout:{"epoch":17,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:34:52.803951+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:34:56.925 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5f84077f70 msgr2=0x7f5f8407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:56.925 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5f84077f70 0x7f5f8407a430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c033000 tx=0x7f5f8c004730 comp rx=0 tx=0).stop 2024-09-10T22:34:56.925 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 msgr2=0x7f5fa40ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:56.925 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40ca8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400c730 tx=0x7f5f9400cbf0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.925 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 shutdown_connections 2024-09-10T22:34:56.925 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5f84077f70 0x7f5f8407a430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.926 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41618b0 0x7f5fa4150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.926 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5fa40d1f20 0x7f5fa40c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.926 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 --2- 172.21.15.66:0/3611463790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fa40b55d0 0x7f5fa40ca8c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:56.926 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 >> 172.21.15.66:0/3611463790 conn(0x7f5fa40d4020 msgr2=0x7f5fa406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:56.926 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 shutdown_connections 2024-09-10T22:34:56.926 INFO:teuthology.misc.health.smithi066.stderr:2024-09-10T22:34:56.924+0000 7f5fa9495640 1 -- 172.21.15.66:0/3611463790 wait complete. 2024-09-10T22:34:56.934 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T22:34:56.934 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T22:34:56.935 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:34:56.935 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:34:56.937 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:34:56.937 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:34:56.938 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:34:56.940 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:34:56.940 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:34:56.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:34:56.942 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:34:56.943 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:34:56.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:34:56.945 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:34:56.945 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T22:34:56.945 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:34:57.014 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:34:57.015 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:34:57.029 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:34:57.030 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:34:57.030 INFO:tasks.ceph.mds.b.smithi094.stderr:2024-09-10T22:34:57.028+0000 7f757a22a640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-10T22:34:57.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.054+0000 7f5b6b857640 1 -- 172.21.15.66:0/2828672266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b6415e090 msgr2=0x7f5b640d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:57.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.054+0000 7f5b6b857640 1 --2- 172.21.15.66:0/2828672266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b6415e090 0x7f5b640d1180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009860 tx=0x7f5b6002a5a0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.054+0000 7f5b6b857640 1 -- 172.21.15.66:0/2828672266 shutdown_connections 2024-09-10T22:34:57.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.054+0000 7f5b6b857640 1 --2- 172.21.15.66:0/2828672266 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b6415e090 0x7f5b640d1180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.054+0000 7f5b6b857640 1 -- 172.21.15.66:0/2828672266 >> 172.21.15.66:0/2828672266 conn(0x7f5b640d3f10 msgr2=0x7f5b640d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:57.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.054+0000 7f5b6b857640 1 -- 172.21.15.66:0/2828672266 shutdown_connections 2024-09-10T22:34:57.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.054+0000 7f5b6b857640 1 -- 172.21.15.66:0/2828672266 wait complete. 2024-09-10T22:34:57.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.055+0000 7f5b6b857640 1 Processor -- start 2024-09-10T22:34:57.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.055+0000 7f5b6b857640 1 -- start start 2024-09-10T22:34:57.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b6b857640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b640d1ae0 0x7f5b641f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:57.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b6b857640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b640d2be0 0x7f5b6414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:57.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b6b857640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b6415e090 0x7f5b640a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:57.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b6b857640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5b640b5030 con 0x7f5b6415e090 2024-09-10T22:34:57.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b6b857640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b640b4eb0 con 0x7f5b640d1ae0 2024-09-10T22:34:57.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b6b857640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5b640b51b0 con 0x7f5b640d2be0 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b6415e090 0x7f5b640a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b6415e090 0x7f5b640a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36700/0 (socket says 172.21.15.66:36700) 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 -- 172.21.15.66:0/3534430559 learned_addr learned my addr 172.21.15.66:0/3534430559 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b695f3640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b640d1ae0 0x7f5b641f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b68df2640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b640d2be0 0x7f5b6414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 -- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b640d2be0 msgr2=0x7f5b6414ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b640d2be0 0x7f5b6414ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 -- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b640d1ae0 msgr2=0x7f5b641f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b640d1ae0 0x7f5b641f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 -- 172.21.15.66:0/3534430559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b64151a40 con 0x7f5b6415e090 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.056+0000 7f5b69df4640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b6415e090 0x7f5b640a06b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c760 tx=0x7f5b5c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.057+0000 7f5b59ffb640 1 -- 172.21.15.66:0/3534430559 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c003b70 con 0x7f5b6415e090 2024-09-10T22:34:57.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.057+0000 7f5b59ffb640 1 -- 172.21.15.66:0/3534430559 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b5c003d30 con 0x7f5b6415e090 2024-09-10T22:34:57.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.057+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b64151d20 con 0x7f5b6415e090 2024-09-10T22:34:57.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.057+0000 7f5b59ffb640 1 -- 172.21.15.66:0/3534430559 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00aab0 con 0x7f5b6415e090 2024-09-10T22:34:57.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.057+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b641522e0 con 0x7f5b6415e090 2024-09-10T22:34:57.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.058+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b640b56e0 con 0x7f5b6415e090 2024-09-10T22:34:57.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.058+0000 7f5b59ffb640 1 -- 172.21.15.66:0/3534430559 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5b5c00ac50 con 0x7f5b6415e090 2024-09-10T22:34:57.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.059+0000 7f5b59ffb640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b48077f70 0x7f5b4807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:57.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.059+0000 7f5b59ffb640 1 -- 172.21.15.66:0/3534430559 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5b5c099640 con 0x7f5b6415e090 2024-09-10T22:34:57.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.059+0000 7f5b695f3640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b48077f70 0x7f5b4807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:57.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.060+0000 7f5b695f3640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b48077f70 0x7f5b4807a430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b60034000 tx=0x7f5b60004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:57.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.063+0000 7f5b59ffb640 1 -- 172.21.15.66:0/3534430559 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5b5c061e50 con 0x7f5b6415e090 2024-09-10T22:34:57.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.290+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b640d1460 con 0x7f5b6415e090 2024-09-10T22:34:57.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.291+0000 7f5b59ffb640 1 -- 172.21.15.66:0/3534430559 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f5b5c066880 con 0x7f5b6415e090 2024-09-10T22:34:57.292 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:57.292 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"btime":"2024-09-10T22:34:57:018547+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:34:57.292 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2024-09-10T22:34:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b48077f70 msgr2=0x7f5b4807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b48077f70 0x7f5b4807a430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b60034000 tx=0x7f5b60004730 comp rx=0 tx=0).stop 2024-09-10T22:34:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b6415e090 msgr2=0x7f5b640a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:57.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b6415e090 0x7f5b640a06b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c760 tx=0x7f5b5c00cc20 comp rx=0 tx=0).stop 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 shutdown_connections 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b48077f70 0x7f5b4807a430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b6415e090 0x7f5b640a06b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5b640d2be0 0x7f5b6414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 --2- 172.21.15.66:0/3534430559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b640d1ae0 0x7f5b641f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 >> 172.21.15.66:0/3534430559 conn(0x7f5b640d3f10 msgr2=0x7f5b640d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 shutdown_connections 2024-09-10T22:34:57.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:57.293+0000 7f5b6b857640 1 -- 172.21.15.66:0/3534430559 wait complete. 2024-09-10T22:34:59.305 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:34:59.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 -- 172.21.15.66:0/355168682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc0cfe30 msgr2=0x7f96dc0cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/355168682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc0cfe30 0x7f96dc0cd3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0d2af0 tx=0x7f96d002a4d0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 -- 172.21.15.66:0/355168682 shutdown_connections 2024-09-10T22:34:59.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/355168682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc0cfe30 0x7f96dc0cd3d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/355168682 >> v2:172.21.15.94:3300/0 conn(0x7f96dc15f290 0x7f96dc0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 -- 172.21.15.66:0/355168682 >> 172.21.15.66:0/355168682 conn(0x7f96dc0d1d20 msgr2=0x7f96dc0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:59.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 -- 172.21.15.66:0/355168682 shutdown_connections 2024-09-10T22:34:59.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.422+0000 7f96e0dcd640 1 -- 172.21.15.66:0/355168682 wait complete. 2024-09-10T22:34:59.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.423+0000 7f96e0dcd640 1 Processor -- start 2024-09-10T22:34:59.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96e0dcd640 1 -- start start 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96e0dcd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc0cd910 0x7f96dc1f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96e0dcd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96dc0cfe30 0x7f96dc06f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96e0dcd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc15f290 0x7f96dc06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96e0dcd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc15f7f0 con 0x7f96dc15f290 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96e0dcd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc15f670 con 0x7f96dc0cd910 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96e0dcd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc0ce3c0 con 0x7f96dc0cfe30 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96dad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc15f290 0x7f96dc06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96dad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc15f290 0x7f96dc06f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36708/0 (socket says 172.21.15.66:36708) 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.424+0000 7f96dad9d640 1 -- 172.21.15.66:0/1279357801 learned_addr learned my addr 172.21.15.66:0/1279357801 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96d9d9b640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96dc0cfe30 0x7f96dc06f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96da59c640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc0cd910 0x7f96dc1f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96dad9d640 1 -- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96dc0cfe30 msgr2=0x7f96dc06f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96dad9d640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96dc0cfe30 0x7f96dc06f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96dad9d640 1 -- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc0cd910 msgr2=0x7f96dc1f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96dad9d640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc0cd910 0x7f96dc1f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96dad9d640 1 -- 172.21.15.66:0/1279357801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc06dcd0 con 0x7f96dc15f290 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96dad9d640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc15f290 0x7f96dc06f7e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c770 tx=0x7f96d400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96caffd640 1 -- 172.21.15.66:0/1279357801 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4017070 con 0x7f96dc15f290 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96caffd640 1 -- 172.21.15.66:0/1279357801 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96d4005070 con 0x7f96dc15f290 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc06df70 con 0x7f96dc15f290 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96caffd640 1 -- 172.21.15.66:0/1279357801 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d40054c0 con 0x7f96dc15f290 2024-09-10T22:34:59.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.425+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc06e530 con 0x7f96dc15f290 2024-09-10T22:34:59.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.426+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc0cd3d0 con 0x7f96dc15f290 2024-09-10T22:34:59.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.427+0000 7f96caffd640 1 -- 172.21.15.66:0/1279357801 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f96d4005660 con 0x7f96dc15f290 2024-09-10T22:34:59.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.428+0000 7f96caffd640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f96bc077f70 0x7f96bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.428+0000 7f96caffd640 1 -- 172.21.15.66:0/1279357801 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f96d40993b0 con 0x7f96dc15f290 2024-09-10T22:34:59.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.428+0000 7f96da59c640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f96bc077f70 0x7f96bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.428+0000 7f96da59c640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f96bc077f70 0x7f96bc07a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009ac0 tx=0x7f96cc005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:59.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.432+0000 7f96caffd640 1 -- 172.21.15.66:0/1279357801 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f96d4062200 con 0x7f96dc15f290 2024-09-10T22:34:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.652+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96dc0cdf90 con 0x7f96dc15f290 2024-09-10T22:34:59.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.652+0000 7f96caffd640 1 -- 172.21.15.66:0/1279357801 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3903 (secure 0 0 0) 0x7f96d4061b00 con 0x7f96dc15f290 2024-09-10T22:34:59.653 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:59.654 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":3,"btime":"2024-09-10T22:34:58:021475+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T22:34:59.654 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 3 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f96bc077f70 msgr2=0x7f96bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f96bc077f70 0x7f96bc07a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009ac0 tx=0x7f96cc005f90 comp rx=0 tx=0).stop 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc15f290 msgr2=0x7f96dc06f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc15f290 0x7f96dc06f7e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c770 tx=0x7f96d400cc30 comp rx=0 tx=0).stop 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 shutdown_connections 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f96bc077f70 0x7f96bc07a430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96dc15f290 0x7f96dc06f7e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f96dc0cfe30 0x7f96dc06f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 --2- 172.21.15.66:0/1279357801 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc0cd910 0x7f96dc1f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 >> 172.21.15.66:0/1279357801 conn(0x7f96dc0d1d20 msgr2=0x7f96dc071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:59.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 shutdown_connections 2024-09-10T22:34:59.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.654+0000 7f96e0dcd640 1 -- 172.21.15.66:0/1279357801 wait complete. 2024-09-10T22:34:59.666 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:34:59.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 -- 172.21.15.66:0/1399207318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac0d2be0 msgr2=0x7fb3ac0d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1399207318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac0d2be0 0x7fb3ac0d1180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac08e690 tx=0x7fb3a002a5f0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 -- 172.21.15.66:0/1399207318 shutdown_connections 2024-09-10T22:34:59.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1399207318 >> v2:172.21.15.94:3300/0 conn(0x7fb3ac0d1ae0 0x7fb3ac0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1399207318 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac0d2be0 0x7fb3ac0d1180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 -- 172.21.15.66:0/1399207318 >> 172.21.15.66:0/1399207318 conn(0x7fb3ac0d3f10 msgr2=0x7fb3ac0d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:59.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 -- 172.21.15.66:0/1399207318 shutdown_connections 2024-09-10T22:34:59.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.770+0000 7fb3b1460640 1 -- 172.21.15.66:0/1399207318 wait complete. 2024-09-10T22:34:59.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.771+0000 7fb3b1460640 1 Processor -- start 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.771+0000 7fb3b1460640 1 -- start start 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3b1460640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ac0d1ae0 0x7fb3ac152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3b1460640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3ac0d2be0 0x7fb3ac0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3b1460640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac15e090 0x7fb3ac0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3b1460640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0b4d20 con 0x7fb3ac15e090 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3b1460640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0b4ba0 con 0x7fb3ac0d1ae0 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3b1460640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac0b4ea0 con 0x7fb3ac0d2be0 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac15e090 0x7fb3ac0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac15e090 0x7fb3ac0cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36724/0 (socket says 172.21.15.66:36724) 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 -- 172.21.15.66:0/1672857763 learned_addr learned my addr 172.21.15.66:0/1672857763 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3aa7fc640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3ac0d2be0 0x7fb3ac0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3aaffd640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ac0d1ae0 0x7fb3ac152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 -- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3ac0d2be0 msgr2=0x7fb3ac0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3ac0d2be0 0x7fb3ac0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 -- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ac0d1ae0 msgr2=0x7fb3ac152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ac0d1ae0 0x7fb3ac152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 -- 172.21.15.66:0/1672857763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac0cc990 con 0x7fb3ac15e090 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3ab7fe640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac15e090 0x7fb3ac0cc160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400db20 tx=0x7fb3a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.772+0000 7fb3937fe640 1 -- 172.21.15.66:0/1672857763 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017070 con 0x7fb3ac15e090 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.773+0000 7fb3937fe640 1 -- 172.21.15.66:0/1672857763 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3a4003970 con 0x7fb3ac15e090 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.773+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac0ccc70 con 0x7fb3ac15e090 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.773+0000 7fb3937fe640 1 -- 172.21.15.66:0/1672857763 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4003dc0 con 0x7fb3ac15e090 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.773+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac0cd230 con 0x7fb3ac15e090 2024-09-10T22:34:59.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.773+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ac0b4450 con 0x7fb3ac15e090 2024-09-10T22:34:59.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.774+0000 7fb3937fe640 1 -- 172.21.15.66:0/1672857763 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb3a4003f60 con 0x7fb3ac15e090 2024-09-10T22:34:59.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.775+0000 7fb3937fe640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb398077f70 0x7fb39807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:34:59.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.775+0000 7fb3937fe640 1 -- 172.21.15.66:0/1672857763 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb3a4099dc0 con 0x7fb3ac15e090 2024-09-10T22:34:59.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.775+0000 7fb3aaffd640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb398077f70 0x7fb39807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:34:59.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.775+0000 7fb3aaffd640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb398077f70 0x7fb39807a430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0b38a0 tx=0x7fb39c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:34:59.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.779+0000 7fb3937fe640 1 -- 172.21.15.66:0/1672857763 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb3a4062650 con 0x7fb3ac15e090 2024-09-10T22:34:59.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.968+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb3ac0d0510 con 0x7fb3ac15e090 2024-09-10T22:34:59.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.969+0000 7fb3937fe640 1 -- 172.21.15.66:0/1672857763 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7fb3a4067080 con 0x7fb3ac15e090 2024-09-10T22:34:59.970 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:34:59.970 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:34:59.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:34:59.970 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:34:59.970 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:34:59.971 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:34:59.971 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:34:59.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb398077f70 msgr2=0x7fb39807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb398077f70 0x7fb39807a430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0b38a0 tx=0x7fb39c009040 comp rx=0 tx=0).stop 2024-09-10T22:34:59.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac15e090 msgr2=0x7fb3ac0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:34:59.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac15e090 0x7fb3ac0cc160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400db20 tx=0x7fb3a400b260 comp rx=0 tx=0).stop 2024-09-10T22:34:59.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 shutdown_connections 2024-09-10T22:34:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb398077f70 0x7fb39807a430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3ac15e090 0x7fb3ac0cc160 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb3ac0d2be0 0x7fb3ac0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 --2- 172.21.15.66:0/1672857763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3ac0d1ae0 0x7fb3ac152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:34:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 >> 172.21.15.66:0/1672857763 conn(0x7fb3ac0d3f10 msgr2=0x7fb3ac06d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:34:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 shutdown_connections 2024-09-10T22:34:59.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:34:59.971+0000 7fb3b1460640 1 -- 172.21.15.66:0/1672857763 wait complete. 2024-09-10T22:34:59.983 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:34:59.983 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.090+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/2561118138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 msgr2=0x7fc3c8162060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.090+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/2561118138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c8162060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80b3690 tx=0x7fc3b802a5f0 comp rx=0 tx=0).stop 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.090+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/2561118138 shutdown_connections 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.090+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/2561118138 >> v2:172.21.15.94:3300/0 conn(0x7fc3c80cfc00 0x7fc3c81625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.090+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/2561118138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c8162060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.090+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/2561118138 >> 172.21.15.66:0/2561118138 conn(0x7fc3c80d3f70 msgr2=0x7fc3c80d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.091+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/2561118138 shutdown_connections 2024-09-10T22:35:00.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.091+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/2561118138 wait complete. 2024-09-10T22:35:00.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.092+0000 7fc3ccb7d640 1 Processor -- start 2024-09-10T22:35:00.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.092+0000 7fc3ccb7d640 1 -- start start 2024-09-10T22:35:00.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c80cfc00 0x7fc3c814b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:00.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c80c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:00.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3c80d2aa0 0x7fc3c80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:00.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c8162ae0 con 0x7fc3c80d26c0 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c80b2f70 con 0x7fc3c80cfc00 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c8162c60 con 0x7fc3c80d2aa0 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c80c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c80c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36742/0 (socket says 172.21.15.66:36742) 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 -- 172.21.15.66:0/568136271 learned_addr learned my addr 172.21.15.66:0/568136271 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c6d9d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3c80d2aa0 0x7fc3c80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c659c640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c80cfc00 0x7fc3c814b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 -- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3c80d2aa0 msgr2=0x7fc3c80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3c80d2aa0 0x7fc3c80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 -- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c80cfc00 msgr2=0x7fc3c814b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c80cfc00 0x7fc3c814b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 -- 172.21.15.66:0/568136271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c80ca8f0 con 0x7fc3c80d26c0 2024-09-10T22:35:00.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c6d9d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3c80d2aa0 0x7fc3c80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c659c640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c80cfc00 0x7fc3c814b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3c5d9b640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c80c7fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e7c0 tx=0x7fc3bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3b6ffd640 1 -- 172.21.15.66:0/568136271 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc00ccc0 con 0x7fc3c80d26c0 2024-09-10T22:35:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3b6ffd640 1 -- 172.21.15.66:0/568136271 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3bc00ce80 con 0x7fc3c80d26c0 2024-09-10T22:35:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3b6ffd640 1 -- 172.21.15.66:0/568136271 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc005280 con 0x7fc3c80d26c0 2024-09-10T22:35:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c80cc130 con 0x7fc3c80d26c0 2024-09-10T22:35:00.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.093+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c80cc6f0 con 0x7fc3c80d26c0 2024-09-10T22:35:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.095+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c8162060 con 0x7fc3c80d26c0 2024-09-10T22:35:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.095+0000 7fc3b6ffd640 1 -- 172.21.15.66:0/568136271 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc3bc005420 con 0x7fc3c80d26c0 2024-09-10T22:35:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.095+0000 7fc3b6ffd640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc3a8077fa0 0x7fc3a807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.096+0000 7fc3b6ffd640 1 -- 172.21.15.66:0/568136271 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc3bc09a310 con 0x7fc3c80d26c0 2024-09-10T22:35:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.096+0000 7fc3c659c640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc3a8077fa0 0x7fc3a807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:00.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.096+0000 7fc3c659c640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc3a8077fa0 0x7fc3a807a460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8033000 tx=0x7fc3b8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:00.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.100+0000 7fc3b6ffd640 1 -- 172.21.15.66:0/568136271 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc3bc0631f0 con 0x7fc3c80d26c0 2024-09-10T22:35:00.284 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:00.283+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc3c815eef0 con 0x7fc3c80d26c0 2024-09-10T22:35:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.037+0000 7fc3b6ffd640 1 -- 172.21.15.66:0/568136271 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fc3bc062af0 con 0x7fc3c80d26c0 2024-09-10T22:35:01.038 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:35:01.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.040+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc3a8077fa0 msgr2=0x7fc3a807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.040+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc3a8077fa0 0x7fc3a807a460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8033000 tx=0x7fc3b8004730 comp rx=0 tx=0).stop 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.040+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 msgr2=0x7fc3c80c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.040+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c80c7fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e7c0 tx=0x7fc3bc00ec80 comp rx=0 tx=0).stop 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 shutdown_connections 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc3a8077fa0 0x7fc3a807a460 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc3c80d2aa0 0x7fc3c80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3c80d26c0 0x7fc3c80c7fd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 --2- 172.21.15.66:0/568136271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3c80cfc00 0x7fc3c814b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 >> 172.21.15.66:0/568136271 conn(0x7fc3c80d3f70 msgr2=0x7fc3c806ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 shutdown_connections 2024-09-10T22:35:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.041+0000 7fc3ccb7d640 1 -- 172.21.15.66:0/568136271 wait complete. 2024-09-10T22:35:01.058 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.175+0000 7fa55dcb0640 1 -- 172.21.15.66:0/1965251110 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5580d10d0 msgr2=0x7fa5580d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.175+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/1965251110 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5580d10d0 0x7fa5580d4a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009820 tx=0x7fa54c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.175+0000 7fa55dcb0640 1 -- 172.21.15.66:0/1965251110 shutdown_connections 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.175+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/1965251110 >> v2:172.21.15.94:3300/0 conn(0x7fa5580d1f90 0x7fa5580d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.175+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/1965251110 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5580d10d0 0x7fa5580d4a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.175+0000 7fa55dcb0640 1 -- 172.21.15.66:0/1965251110 >> 172.21.15.66:0/1965251110 conn(0x7fa5580d3f10 msgr2=0x7fa5580d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.176+0000 7fa55dcb0640 1 -- 172.21.15.66:0/1965251110 shutdown_connections 2024-09-10T22:35:01.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.176+0000 7fa55dcb0640 1 -- 172.21.15.66:0/1965251110 wait complete. 2024-09-10T22:35:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.177+0000 7fa55dcb0640 1 Processor -- start 2024-09-10T22:35:01.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.177+0000 7fa55dcb0640 1 -- start start 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa55dcb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5580d10d0 0x7fa55814c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa55dcb0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5580d1f90 0x7fa558159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa55dcb0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa55815e120 0x7fa55815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa55dcb0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580b70f0 con 0x7fa5580d1f90 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa55dcb0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580b6f70 con 0x7fa5580d10d0 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa55dcb0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa5580b7270 con 0x7fa55815e120 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5580d1f90 0x7fa558159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5580d1f90 0x7fa558159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36758/0 (socket says 172.21.15.66:36758) 2024-09-10T22:35:01.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 -- 172.21.15.66:0/3552981731 learned_addr learned my addr 172.21.15.66:0/3552981731 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa557fff640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa55815e120 0x7fa55815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa5577fe640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5580d10d0 0x7fa55814c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 -- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa55815e120 msgr2=0x7fa55815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa55815e120 0x7fa55815a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 -- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5580d10d0 msgr2=0x7fa55814c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5580d10d0 0x7fa55814c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 -- 172.21.15.66:0/3552981731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa558154e30 con 0x7fa5580d1f90 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa556ffd640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5580d1f90 0x7fa558159c00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009940 tx=0x7fa54c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.178+0000 7fa53ffff640 1 -- 172.21.15.66:0/3552981731 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c040070 con 0x7fa5580d1f90 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.179+0000 7fa53ffff640 1 -- 172.21.15.66:0/3552981731 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa54c004e50 con 0x7fa5580d1f90 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.179+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55814c8e0 con 0x7fa5580d1f90 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.179+0000 7fa53ffff640 1 -- 172.21.15.66:0/3552981731 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0055b0 con 0x7fa5580d1f90 2024-09-10T22:35:01.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.179+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55815a9d0 con 0x7fa5580d1f90 2024-09-10T22:35:01.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.180+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5580b7f00 con 0x7fa5580d1f90 2024-09-10T22:35:01.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.180+0000 7fa53ffff640 1 -- 172.21.15.66:0/3552981731 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa54c03c420 con 0x7fa5580d1f90 2024-09-10T22:35:01.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.181+0000 7fa53ffff640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa544077f70 0x7fa54407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:01.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.181+0000 7fa53ffff640 1 -- 172.21.15.66:0/3552981731 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fa54c0bb360 con 0x7fa5580d1f90 2024-09-10T22:35:01.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.181+0000 7fa5577fe640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa544077f70 0x7fa54407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:01.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.182+0000 7fa5577fe640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa544077f70 0x7fa54407a430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa548001720 tx=0x7fa548009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:01.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.186+0000 7fa53ffff640 1 -- 172.21.15.66:0/3552981731 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa54c083970 con 0x7fa5580d1f90 2024-09-10T22:35:01.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:01.376+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa5580cfa80 con 0x7fa5580d1f90 2024-09-10T22:35:01.517 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:35:01.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:35:02.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.043+0000 7fa53ffff640 1 -- 172.21.15.66:0/3552981731 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7fa54c0883a0 con 0x7fa5580d1f90 2024-09-10T22:35:02.044 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:35:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.050+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa544077f70 msgr2=0x7fa54407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.050+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa544077f70 0x7fa54407a430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa548001720 tx=0x7fa548009040 comp rx=0 tx=0).stop 2024-09-10T22:35:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.050+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5580d1f90 msgr2=0x7fa558159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:02.051 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.050+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5580d1f90 0x7fa558159c00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009940 tx=0x7fa54c004a50 comp rx=0 tx=0).stop 2024-09-10T22:35:02.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.053+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 shutdown_connections 2024-09-10T22:35:02.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.053+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa544077f70 0x7fa54407a430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.053+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa55815e120 0x7fa55815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.053+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa5580d1f90 0x7fa558159c00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.053+0000 7fa55dcb0640 1 --2- 172.21.15.66:0/3552981731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5580d10d0 0x7fa55814c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.053+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 >> 172.21.15.66:0/3552981731 conn(0x7fa5580d3f10 msgr2=0x7fa55806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:02.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.055+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 shutdown_connections 2024-09-10T22:35:02.058 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.055+0000 7fa55dcb0640 1 -- 172.21.15.66:0/3552981731 wait complete. 2024-09-10T22:35:02.075 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:35:02.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 -- 172.21.15.66:0/1840214549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c0b2d70 msgr2=0x7f1e3c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:02.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 --2- 172.21.15.66:0/1840214549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c0b2d70 0x7f1e3c0b1bc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009830 tx=0x7f1e3002a510 comp rx=0 tx=0).stop 2024-09-10T22:35:02.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 -- 172.21.15.66:0/1840214549 shutdown_connections 2024-09-10T22:35:02.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 --2- 172.21.15.66:0/1840214549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c0b2d70 0x7f1e3c0b1bc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 --2- 172.21.15.66:0/1840214549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e3c0af490 0x7f1e3c0af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 -- 172.21.15.66:0/1840214549 >> 172.21.15.66:0/1840214549 conn(0x7f1e3c0d1db0 msgr2=0x7f1e3c0d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:02.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 -- 172.21.15.66:0/1840214549 shutdown_connections 2024-09-10T22:35:02.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.185+0000 7f1e41272640 1 -- 172.21.15.66:0/1840214549 wait complete. 2024-09-10T22:35:02.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.186+0000 7f1e41272640 1 Processor -- start 2024-09-10T22:35:02.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.186+0000 7f1e41272640 1 -- start start 2024-09-10T22:35:02.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e41272640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e3c0af490 0x7f1e3c1532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:02.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e41272640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e3c0b2d70 0x7f1e3c0b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:02.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e41272640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c15f5b0 0x7f1e3c0b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:02.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e41272640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c0b33d0 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e41272640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c0b2310 con 0x7f1e3c0af490 2024-09-10T22:35:02.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e41272640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3c0afe60 con 0x7f1e3c0b2d70 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c15f5b0 0x7f1e3c0b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c15f5b0 0x7f1e3c0b7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36770/0 (socket says 172.21.15.66:36770) 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 -- 172.21.15.66:0/4066204926 learned_addr learned my addr 172.21.15.66:0/4066204926 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3affd640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e3c0af490 0x7f1e3c1532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3a7fc640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e3c0b2d70 0x7f1e3c0b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 -- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e3c0b2d70 msgr2=0x7f1e3c0b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e3c0b2d70 0x7f1e3c0b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 -- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e3c0af490 msgr2=0x7f1e3c1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e3c0af490 0x7f1e3c1532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 -- 172.21.15.66:0/4066204926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c153810 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3a7fc640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e3c0b2d70 0x7f1e3c0b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.187+0000 7f1e3b7fe640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c15f5b0 0x7f1e3c0b7270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400cce0 tx=0x7f1e3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.188+0000 7f1e237fe640 1 -- 172.21.15.66:0/4066204926 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400f450 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.188+0000 7f1e237fe640 1 -- 172.21.15.66:0/4066204926 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e3400f700 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.188+0000 7f1e237fe640 1 -- 172.21.15.66:0/4066204926 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400fb50 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.188+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3c1f2c80 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.188+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e3c1f3240 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.189+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e3c0af920 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.189+0000 7f1e237fe640 1 -- 172.21.15.66:0/4066204926 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1e340213f0 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.190+0000 7f1e237fe640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1e28077f70 0x7f1e2807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.190+0000 7f1e237fe640 1 -- 172.21.15.66:0/4066204926 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f1e3409a900 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.190+0000 7f1e3affd640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1e28077f70 0x7f1e2807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:02.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.191+0000 7f1e3affd640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1e28077f70 0x7f1e2807a430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0ce0e0 tx=0x7f1e2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:02.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.195+0000 7f1e237fe640 1 -- 172.21.15.66:0/4066204926 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1e3c153810 con 0x7f1e3c15f5b0 2024-09-10T22:35:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:02.418+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1e3c0b0cd0 con 0x7f1e3c15f5b0 2024-09-10T22:35:03.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.052+0000 7f1e237fe640 1 -- 172.21.15.66:0/4066204926 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f1e34014030 con 0x7f1e3c15f5b0 2024-09-10T22:35:03.053 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T22:35:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.058+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1e28077f70 msgr2=0x7f1e2807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.058+0000 7f1e41272640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1e28077f70 0x7f1e2807a430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0ce0e0 tx=0x7f1e2c009040 comp rx=0 tx=0).stop 2024-09-10T22:35:03.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.058+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c15f5b0 msgr2=0x7f1e3c0b7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.058+0000 7f1e41272640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c15f5b0 0x7f1e3c0b7270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400cce0 tx=0x7f1e3400a620 comp rx=0 tx=0).stop 2024-09-10T22:35:03.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.060+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 shutdown_connections 2024-09-10T22:35:03.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.060+0000 7f1e41272640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1e28077f70 0x7f1e2807a430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.060+0000 7f1e41272640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1e3c15f5b0 0x7f1e3c0b7270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.060+0000 7f1e41272640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1e3c0b2d70 0x7f1e3c0b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.060+0000 7f1e41272640 1 --2- 172.21.15.66:0/4066204926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e3c0af490 0x7f1e3c1532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.060+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 >> 172.21.15.66:0/4066204926 conn(0x7f1e3c0d1db0 msgr2=0x7f1e3c0c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.063+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 shutdown_connections 2024-09-10T22:35:03.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.063+0000 7f1e41272640 1 -- 172.21.15.66:0/4066204926 wait complete. 2024-09-10T22:35:03.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.194+0000 7f204767f640 1 -- 172.21.15.66:0/2858773974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d2be0 msgr2=0x7f20400d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.194+0000 7f204767f640 1 --2- 172.21.15.66:0/2858773974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d2be0 0x7f20400d1180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2034009860 tx=0x7f203402a530 comp rx=0 tx=0).stop 2024-09-10T22:35:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.195+0000 7f204767f640 1 -- 172.21.15.66:0/2858773974 shutdown_connections 2024-09-10T22:35:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.195+0000 7f204767f640 1 --2- 172.21.15.66:0/2858773974 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d2be0 0x7f20400d1180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.195+0000 7f204767f640 1 -- 172.21.15.66:0/2858773974 >> 172.21.15.66:0/2858773974 conn(0x7f20400d3f10 msgr2=0x7f20400d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.195+0000 7f204767f640 1 -- 172.21.15.66:0/2858773974 shutdown_connections 2024-09-10T22:35:03.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.195+0000 7f204767f640 1 -- 172.21.15.66:0/2858773974 wait complete. 2024-09-10T22:35:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.196+0000 7f204767f640 1 Processor -- start 2024-09-10T22:35:03.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.196+0000 7f204767f640 1 -- start start 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204767f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d1ae0 0x7f20400b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204767f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20400d2be0 0x7f20401f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204767f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204015e090 0x7f20400b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204767f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f20400b4fb0 con 0x7f20400d2be0 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204767f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20400b4e30 con 0x7f204015e090 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204767f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f20400b5130 con 0x7f20400d1ae0 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20400d2be0 0x7f20401f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20400d2be0 0x7f20401f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36786/0 (socket says 172.21.15.66:36786) 2024-09-10T22:35:03.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 -- 172.21.15.66:0/800937770 learned_addr learned my addr 172.21.15.66:0/800937770 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204541b640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d1ae0 0x7f20400b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2045c1c640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204015e090 0x7f20400b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 -- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d1ae0 msgr2=0x7f20400b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d1ae0 0x7f20400b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 -- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204015e090 msgr2=0x7f20400b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204015e090 0x7f20400b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 -- 172.21.15.66:0/800937770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2040152c50 con 0x7f20400d2be0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f2044c1a640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20400d2be0 0x7f20401f3d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2034009980 tx=0x7f2034003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f203dffb640 1 -- 172.21.15.66:0/800937770 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034005c90 con 0x7f20400d2be0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f203dffb640 1 -- 172.21.15.66:0/800937770 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2034005020 con 0x7f20400d2be0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2040152f30 con 0x7f20400d2be0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.197+0000 7f203dffb640 1 -- 172.21.15.66:0/800937770 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034005410 con 0x7f20400d2be0 2024-09-10T22:35:03.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.198+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401534f0 con 0x7f20400d2be0 2024-09-10T22:35:03.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.199+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20400b5620 con 0x7f20400d2be0 2024-09-10T22:35:03.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.199+0000 7f203dffb640 1 -- 172.21.15.66:0/800937770 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f20340317c0 con 0x7f20400d2be0 2024-09-10T22:35:03.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.200+0000 7f203dffb640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f202c077f70 0x7f202c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.200+0000 7f203dffb640 1 -- 172.21.15.66:0/800937770 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f20340bb070 con 0x7f20400d2be0 2024-09-10T22:35:03.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.200+0000 7f204541b640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f202c077f70 0x7f202c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.200+0000 7f204541b640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f202c077f70 0x7f202c07a430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20300015c0 tx=0x7f2030009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:03.205 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.204+0000 7f203dffb640 1 -- 172.21.15.66:0/800937770 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2034040030 con 0x7f20400d2be0 2024-09-10T22:35:03.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.389+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20400d2650 con 0x7f20400d2be0 2024-09-10T22:35:03.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.390+0000 7f203dffb640 1 -- 172.21.15.66:0/800937770 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16654 (secure 0 0 0) 0x7f2034083af0 con 0x7f20400d2be0 2024-09-10T22:35:03.392 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:03.392 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":20,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:02.418889+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f202c077f70 msgr2=0x7f202c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f202c077f70 0x7f202c07a430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20300015c0 tx=0x7f2030009040 comp rx=0 tx=0).stop 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20400d2be0 msgr2=0x7f20401f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20400d2be0 0x7f20401f3d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2034009980 tx=0x7f2034003c10 comp rx=0 tx=0).stop 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 shutdown_connections 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f202c077f70 0x7f202c07a430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f204015e090 0x7f20400b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20400d2be0 0x7f20401f3d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 --2- 172.21.15.66:0/800937770 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f20400d1ae0 0x7f20400b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.392+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 >> 172.21.15.66:0/800937770 conn(0x7f20400d3f10 msgr2=0x7f204006e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:03.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.393+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 shutdown_connections 2024-09-10T22:35:03.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.393+0000 7f204767f640 1 -- 172.21.15.66:0/800937770 wait complete. 2024-09-10T22:35:03.404 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 -- 172.21.15.66:0/2399099006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d2be0 msgr2=0x7fd1480d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/2399099006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d2be0 0x7fd1480d1180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd13c005fd0 tx=0x7fd13c02a5d0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 -- 172.21.15.66:0/2399099006 shutdown_connections 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/2399099006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d2be0 0x7fd1480d1180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/2399099006 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1480d1ae0 0x7fd1480b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 -- 172.21.15.66:0/2399099006 >> 172.21.15.66:0/2399099006 conn(0x7fd1480d3f10 msgr2=0x7fd1480d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 -- 172.21.15.66:0/2399099006 shutdown_connections 2024-09-10T22:35:03.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.507+0000 7fd14ff6b640 1 -- 172.21.15.66:0/2399099006 wait complete. 2024-09-10T22:35:03.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.508+0000 7fd14ff6b640 1 Processor -- start 2024-09-10T22:35:03.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14ff6b640 1 -- start start 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14ff6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d1ae0 0x7fd1480b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14ff6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1480d2be0 0x7fd1480ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14ff6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd14815e090 0x7fd1481f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14ff6b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd1480b4d20 con 0x7fd1480d1ae0 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14ff6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1480b4ba0 con 0x7fd14815e090 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14ff6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd1480b4ea0 con 0x7fd1480d2be0 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14dd07640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d1ae0 0x7fd1480b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14dd07640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d1ae0 0x7fd1480b9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55586/0 (socket says 172.21.15.66:55586) 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.509+0000 7fd14dd07640 1 -- 172.21.15.66:0/155336350 learned_addr learned my addr 172.21.15.66:0/155336350 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:03.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14e508640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd14815e090 0x7fd1481f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14d506640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1480d2be0 0x7fd1480ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14dd07640 1 -- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1480d2be0 msgr2=0x7fd1480ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14dd07640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1480d2be0 0x7fd1480ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14dd07640 1 -- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd14815e090 msgr2=0x7fd1481f7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14dd07640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd14815e090 0x7fd1481f7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14dd07640 1 -- 172.21.15.66:0/155336350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1480b8c80 con 0x7fd1480d1ae0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14e508640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd14815e090 0x7fd1481f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14d506640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1480d2be0 0x7fd1480ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14dd07640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d1ae0 0x7fd1480b9720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c3c0 tx=0x7fd14000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd1467fc640 1 -- 172.21.15.66:0/155336350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140017070 con 0x7fd1480d1ae0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd1467fc640 1 -- 172.21.15.66:0/155336350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1400035f0 con 0x7fd1480d1ae0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1481f87c0 con 0x7fd1480d1ae0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd1467fc640 1 -- 172.21.15.66:0/155336350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140003a80 con 0x7fd1480d1ae0 2024-09-10T22:35:03.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.510+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1480b8f20 con 0x7fd1480d1ae0 2024-09-10T22:35:03.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.511+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1480b65b0 con 0x7fd1480d1ae0 2024-09-10T22:35:03.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.512+0000 7fd1467fc640 1 -- 172.21.15.66:0/155336350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd140003c20 con 0x7fd1480d1ae0 2024-09-10T22:35:03.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.512+0000 7fd1467fc640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd134077fa0 0x7fd13407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.514 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.512+0000 7fd1467fc640 1 -- 172.21.15.66:0/155336350 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd1400992e0 con 0x7fd1480d1ae0 2024-09-10T22:35:03.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.512+0000 7fd14d506640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd134077fa0 0x7fd13407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.515 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.513+0000 7fd14d506640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd134077fa0 0x7fd13407a460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd13c033000 tx=0x7fd13c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:03.517 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.516+0000 7fd1467fc640 1 -- 172.21.15.66:0/155336350 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd140061de0 con 0x7fd1480d1ae0 2024-09-10T22:35:03.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.701+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1480b1d60 con 0x7fd1480d1ae0 2024-09-10T22:35:03.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.702+0000 7fd1467fc640 1 -- 172.21.15.66:0/155336350 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16654 (secure 0 0 0) 0x7fd1400616e0 con 0x7fd1480d1ae0 2024-09-10T22:35:03.703 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:03.703 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":20,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:02.418889+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:03.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd134077fa0 msgr2=0x7fd13407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd134077fa0 0x7fd13407a460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd13c033000 tx=0x7fd13c004730 comp rx=0 tx=0).stop 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d1ae0 msgr2=0x7fd1480b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d1ae0 0x7fd1480b9720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd14000c3c0 tx=0x7fd14000c880 comp rx=0 tx=0).stop 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 shutdown_connections 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd134077fa0 0x7fd13407a460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd14815e090 0x7fd1481f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1480d2be0 0x7fd1480ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 --2- 172.21.15.66:0/155336350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1480d1ae0 0x7fd1480b9720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 >> 172.21.15.66:0/155336350 conn(0x7fd1480d3f10 msgr2=0x7fd1480d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 shutdown_connections 2024-09-10T22:35:03.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.704+0000 7fd14ff6b640 1 -- 172.21.15.66:0/155336350 wait complete. 2024-09-10T22:35:03.716 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:35:03.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.822+0000 7f8f46105640 1 -- 172.21.15.66:0/1074380019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f400b2d70 msgr2=0x7f8f400b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.822+0000 7f8f46105640 1 --2- 172.21.15.66:0/1074380019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f400b2d70 0x7f8f400b1bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009830 tx=0x7f8f3402a510 comp rx=0 tx=0).stop 2024-09-10T22:35:03.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.822+0000 7f8f46105640 1 -- 172.21.15.66:0/1074380019 shutdown_connections 2024-09-10T22:35:03.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.822+0000 7f8f46105640 1 --2- 172.21.15.66:0/1074380019 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f400b2d70 0x7f8f400b1bc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.822+0000 7f8f46105640 1 -- 172.21.15.66:0/1074380019 >> 172.21.15.66:0/1074380019 conn(0x7f8f400d1db0 msgr2=0x7f8f400d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:03.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.822+0000 7f8f46105640 1 -- 172.21.15.66:0/1074380019 shutdown_connections 2024-09-10T22:35:03.824 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.822+0000 7f8f46105640 1 -- 172.21.15.66:0/1074380019 wait complete. 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.824+0000 7f8f46105640 1 Processor -- start 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.824+0000 7f8f46105640 1 -- start start 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.824+0000 7f8f46105640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f400af490 0x7f8f400b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.824+0000 7f8f46105640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f400b2d70 0x7f8f400b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f46105640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f4015f5b0 0x7f8f401f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f46105640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400b0010 con 0x7f8f4015f5b0 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f46105640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400b2310 con 0x7f8f400b2d70 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f46105640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8f400b0190 con 0x7f8f400af490 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f4015f5b0 0x7f8f401f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f4015f5b0 0x7f8f401f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55604/0 (socket says 172.21.15.66:55604) 2024-09-10T22:35:03.827 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 -- 172.21.15.66:0/2420565549 learned_addr learned my addr 172.21.15.66:0/2420565549 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3f7fe640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f400af490 0x7f8f400b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3effd640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f400b2d70 0x7f8f400b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 -- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f400af490 msgr2=0x7f8f400b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f400af490 0x7f8f400b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 -- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f400b2d70 msgr2=0x7f8f400b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f400b2d70 0x7f8f400b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 -- 172.21.15.66:0/2420565549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4006dc30 con 0x7f8f4015f5b0 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3effd640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f400b2d70 0x7f8f400b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f3ffff640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f4015f5b0 0x7f8f401f4af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800cbc0 tx=0x7f8f3800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f27fff640 1 -- 172.21.15.66:0/2420565549 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38016020 con 0x7f8f4015f5b0 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f27fff640 1 -- 172.21.15.66:0/2420565549 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f3800ae30 con 0x7f8f4015f5b0 2024-09-10T22:35:03.828 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f401f53b0 con 0x7f8f4015f5b0 2024-09-10T22:35:03.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f27fff640 1 -- 172.21.15.66:0/2420565549 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38012660 con 0x7f8f4015f5b0 2024-09-10T22:35:03.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.825+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401f5970 con 0x7f8f4015f5b0 2024-09-10T22:35:03.829 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.826+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f400b0680 con 0x7f8f4015f5b0 2024-09-10T22:35:03.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.827+0000 7f8f27fff640 1 -- 172.21.15.66:0/2420565549 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8f3801a3b0 con 0x7f8f4015f5b0 2024-09-10T22:35:03.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.827+0000 7f8f27fff640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8f2c077fa0 0x7f8f2c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:03.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.828+0000 7f8f27fff640 1 -- 172.21.15.66:0/2420565549 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f3809a310 con 0x7f8f4015f5b0 2024-09-10T22:35:03.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.828+0000 7f8f3f7fe640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8f2c077fa0 0x7f8f2c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:03.830 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.828+0000 7f8f3f7fe640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8f2c077fa0 0x7f8f2c07a460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006fd0 tx=0x7f8f30009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:03.833 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:03.832+0000 7f8f27fff640 1 -- 172.21.15.66:0/2420565549 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8f38020030 con 0x7f8f4015f5b0 2024-09-10T22:35:04.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:04.055+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8f4015d530 con 0x7f8f4015f5b0 2024-09-10T22:35:05.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.063+0000 7f8f27fff640 1 -- 172.21.15.66:0/2420565549 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f8f38062e10 con 0x7f8f4015f5b0 2024-09-10T22:35:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8f2c077fa0 msgr2=0x7f8f2c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8f2c077fa0 0x7f8f2c07a460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8f30006fd0 tx=0x7f8f30009060 comp rx=0 tx=0).stop 2024-09-10T22:35:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f4015f5b0 msgr2=0x7f8f401f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f4015f5b0 0x7f8f401f4af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800cbc0 tx=0x7f8f3800a430 comp rx=0 tx=0).stop 2024-09-10T22:35:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 shutdown_connections 2024-09-10T22:35:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8f2c077fa0 0x7f8f2c07a460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f4015f5b0 0x7f8f401f4af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f400b2d70 0x7f8f400b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 --2- 172.21.15.66:0/2420565549 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8f400af490 0x7f8f400b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 >> 172.21.15.66:0/2420565549 conn(0x7f8f400d1db0 msgr2=0x7f8f40072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:05.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 shutdown_connections 2024-09-10T22:35:05.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.066+0000 7f8f46105640 1 -- 172.21.15.66:0/2420565549 wait complete. 2024-09-10T22:35:05.085 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:35:05.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:35:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 -- 172.21.15.66:0/3914467175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c815e120 msgr2=0x7f37c80d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:05.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 --2- 172.21.15.66:0/3914467175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c815e120 0x7f37c80d4a30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37c4005fd0 tx=0x7f37c402a4b0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 -- 172.21.15.66:0/3914467175 shutdown_connections 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 --2- 172.21.15.66:0/3914467175 >> v2:172.21.15.94:3300/0 conn(0x7f37c80d10d0 0x7f37c80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 --2- 172.21.15.66:0/3914467175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c815e120 0x7f37c80d4a30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 -- 172.21.15.66:0/3914467175 >> 172.21.15.66:0/3914467175 conn(0x7f37c80d3f10 msgr2=0x7f37c80d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 -- 172.21.15.66:0/3914467175 shutdown_connections 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.207+0000 7f37cf94b640 1 -- 172.21.15.66:0/3914467175 wait complete. 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.208+0000 7f37cf94b640 1 Processor -- start 2024-09-10T22:35:05.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.208+0000 7f37cf94b640 1 -- start start 2024-09-10T22:35:05.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cf94b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80d10d0 0x7f37c81f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:05.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cf94b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80d1f90 0x7f37c815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:05.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cf94b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c815e120 0x7f37c8160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cf94b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80b7800 con 0x7f37c815e120 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cf94b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80b7680 con 0x7f37c80d10d0 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cf94b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f37c80d2440 con 0x7f37c80d1f90 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c815e120 0x7f37c8160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37ccee6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80d1f90 0x7f37c815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c815e120 0x7f37c8160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55620/0 (socket says 172.21.15.66:55620) 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 -- 172.21.15.66:0/1976742922 learned_addr learned my addr 172.21.15.66:0/1976742922 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cd6e7640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80d10d0 0x7f37c81f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 -- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80d1f90 msgr2=0x7f37c815afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80d1f90 0x7f37c815afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 -- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80d10d0 msgr2=0x7f37c81f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80d10d0 0x7f37c81f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:05.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.209+0000 7f37cdee8640 1 -- 172.21.15.66:0/1976742922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c80cdac0 con 0x7f37c815e120 2024-09-10T22:35:05.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.210+0000 7f37cdee8640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c815e120 0x7f37c8160bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c790 tx=0x7f37c000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:05.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.210+0000 7f37b9ffb640 1 -- 172.21.15.66:0/1976742922 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0003b70 con 0x7f37c815e120 2024-09-10T22:35:05.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.210+0000 7f37b9ffb640 1 -- 172.21.15.66:0/1976742922 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37c0003d30 con 0x7f37c815e120 2024-09-10T22:35:05.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.210+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c80cdda0 con 0x7f37c815e120 2024-09-10T22:35:05.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.210+0000 7f37b9ffb640 1 -- 172.21.15.66:0/1976742922 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c000aab0 con 0x7f37c815e120 2024-09-10T22:35:05.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.210+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81603b0 con 0x7f37c815e120 2024-09-10T22:35:05.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.211+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c80cf850 con 0x7f37c815e120 2024-09-10T22:35:05.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.212+0000 7f37b9ffb640 1 -- 172.21.15.66:0/1976742922 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f37c000ac50 con 0x7f37c815e120 2024-09-10T22:35:05.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.212+0000 7f37b9ffb640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f37ac077f70 0x7f37ac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:05.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.212+0000 7f37b9ffb640 1 -- 172.21.15.66:0/1976742922 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f37c0099a70 con 0x7f37c815e120 2024-09-10T22:35:05.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.213+0000 7f37cd6e7640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f37ac077f70 0x7f37ac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:05.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.213+0000 7f37cd6e7640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f37ac077f70 0x7f37ac07a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f37c4033000 tx=0x7f37c4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:05.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.216+0000 7f37b9ffb640 1 -- 172.21.15.66:0/1976742922 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f37c0061f20 con 0x7f37c815e120 2024-09-10T22:35:05.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:05.439+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f37c80b9c40 con 0x7f37c815e120 2024-09-10T22:35:06.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.068+0000 7f37b9ffb640 1 -- 172.21.15.66:0/1976742922 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f37c0066950 con 0x7f37c815e120 2024-09-10T22:35:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f37ac077f70 msgr2=0x7f37ac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f37ac077f70 0x7f37ac07a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f37c4033000 tx=0x7f37c4004730 comp rx=0 tx=0).stop 2024-09-10T22:35:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c815e120 msgr2=0x7f37c8160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c815e120 0x7f37c8160bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c790 tx=0x7f37c000cc50 comp rx=0 tx=0).stop 2024-09-10T22:35:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 shutdown_connections 2024-09-10T22:35:06.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f37ac077f70 0x7f37ac07a430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37c815e120 0x7f37c8160bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f37c80d1f90 0x7f37c815afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 --2- 172.21.15.66:0/1976742922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37c80d10d0 0x7f37c81f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 >> 172.21.15.66:0/1976742922 conn(0x7f37c80d3f10 msgr2=0x7f37c80b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 shutdown_connections 2024-09-10T22:35:06.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.071+0000 7f37cf94b640 1 -- 172.21.15.66:0/1976742922 wait complete. 2024-09-10T22:35:06.084 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:35:06.084 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.192+0000 7f474ff5d640 1 -- 172.21.15.66:0/3944549426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4734003c30 msgr2=0x7f4734004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.192+0000 7f474ff5d640 1 --2- 172.21.15.66:0/3944549426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4734003c30 0x7f4734004030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4744009b50 tx=0x7f4734005830 comp rx=0 tx=0).stop 2024-09-10T22:35:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.192+0000 7f474ff5d640 1 -- 172.21.15.66:0/3944549426 shutdown_connections 2024-09-10T22:35:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.192+0000 7f474ff5d640 1 --2- 172.21.15.66:0/3944549426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4734003c30 0x7f4734004030 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.192+0000 7f474ff5d640 1 -- 172.21.15.66:0/3944549426 >> 172.21.15.66:0/3944549426 conn(0x7f47480d1d20 msgr2=0x7f47480d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.192+0000 7f474ff5d640 1 -- 172.21.15.66:0/3944549426 shutdown_connections 2024-09-10T22:35:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.192+0000 7f474ff5d640 1 -- 172.21.15.66:0/3944549426 wait complete. 2024-09-10T22:35:06.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 Processor -- start 2024-09-10T22:35:06.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 -- start start 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4734003c30 0x7f47480ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47480b5560 0x7f47481f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47480b59e0 0x7f4748071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f47480ce3c0 con 0x7f47480b5560 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47480cf640 con 0x7f47480b59e0 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.194+0000 7f474ff5d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f47480ce540 con 0x7f4734003c30 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47480b5560 0x7f47481f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47480b5560 0x7f47481f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55640/0 (socket says 172.21.15.66:55640) 2024-09-10T22:35:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 -- 172.21.15.66:0/1889140319 learned_addr learned my addr 172.21.15.66:0/1889140319 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474dcf9640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4734003c30 0x7f47480ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474e4fa640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47480b59e0 0x7f4748071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 -- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4734003c30 msgr2=0x7f47480ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4734003c30 0x7f47480ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 -- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47480b59e0 msgr2=0x7f4748071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47480b59e0 0x7f4748071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 -- 172.21.15.66:0/1889140319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47481f6b00 con 0x7f47480b5560 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474d4f8640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47480b5560 0x7f47481f6040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f473800e5c0 tx=0x7f473800ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f473e7fc640 1 -- 172.21.15.66:0/1889140319 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738015070 con 0x7f47480b5560 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f473e7fc640 1 -- 172.21.15.66:0/1889140319 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47380108b0 con 0x7f47480b5560 2024-09-10T22:35:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47481f6de0 con 0x7f47480b5560 2024-09-10T22:35:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f473e7fc640 1 -- 172.21.15.66:0/1889140319 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738010d00 con 0x7f47480b5560 2024-09-10T22:35:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.195+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47480b5040 con 0x7f47480b5560 2024-09-10T22:35:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.196+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47480b4b50 con 0x7f47480b5560 2024-09-10T22:35:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.197+0000 7f473e7fc640 1 -- 172.21.15.66:0/1889140319 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4738011460 con 0x7f47480b5560 2024-09-10T22:35:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.198+0000 7f473e7fc640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f472c077fa0 0x7f472c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.198+0000 7f473e7fc640 1 -- 172.21.15.66:0/1889140319 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f473809a250 con 0x7f47480b5560 2024-09-10T22:35:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.198+0000 7f474dcf9640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f472c077fa0 0x7f472c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.198+0000 7f474dcf9640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f472c077fa0 0x7f472c07a460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4744035000 tx=0x7f47440046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.202+0000 7f473e7fc640 1 -- 172.21.15.66:0/1889140319 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4738062700 con 0x7f47480b5560 2024-09-10T22:35:06.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.423+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47480b5210 con 0x7f47480b5560 2024-09-10T22:35:06.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.424+0000 7f473e7fc640 1 -- 172.21.15.66:0/1889140319 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f4738067130 con 0x7f47480b5560 2024-09-10T22:35:06.425 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:06.425 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:06.425 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:06.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f472c077fa0 msgr2=0x7f472c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f472c077fa0 0x7f472c07a460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4744035000 tx=0x7f47440046c0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47480b5560 msgr2=0x7f47481f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47480b5560 0x7f47481f6040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f473800e5c0 tx=0x7f473800ea80 comp rx=0 tx=0).stop 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 shutdown_connections 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f472c077fa0 0x7f472c07a460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47480b59e0 0x7f4748071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47480b5560 0x7f47481f6040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 --2- 172.21.15.66:0/1889140319 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f4734003c30 0x7f47480ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 >> 172.21.15.66:0/1889140319 conn(0x7f47480d1d20 msgr2=0x7f47480ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 shutdown_connections 2024-09-10T22:35:06.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.426+0000 7f474ff5d640 1 -- 172.21.15.66:0/1889140319 wait complete. 2024-09-10T22:35:06.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:06.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 -- 172.21.15.66:0/1079222160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb10015be30 msgr2=0x7fb1000688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 --2- 172.21.15.66:0/1079222160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb10015be30 0x7fb1000688e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb1000caa10 tx=0x7fb0f802a580 comp rx=0 tx=0).stop 2024-09-10T22:35:06.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 -- 172.21.15.66:0/1079222160 shutdown_connections 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 --2- 172.21.15.66:0/1079222160 >> v2:172.21.15.94:3300/0 conn(0x7fb1000b1480 0x7fb1000741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 --2- 172.21.15.66:0/1079222160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb100160350 0x7fb1000b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 --2- 172.21.15.66:0/1079222160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb10015be30 0x7fb1000688e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 -- 172.21.15.66:0/1079222160 >> 172.21.15.66:0/1079222160 conn(0x7fb1000d0430 msgr2=0x7fb1000d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 -- 172.21.15.66:0/1079222160 shutdown_connections 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.541+0000 7fb10809b640 1 -- 172.21.15.66:0/1079222160 wait complete. 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.542+0000 7fb10809b640 1 Processor -- start 2024-09-10T22:35:06.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.542+0000 7fb10809b640 1 -- start start 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb10809b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1000b1480 0x7fb1001f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb10809b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb10015be30 0x7fb1000b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb10809b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb100160350 0x7fb1000c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb10809b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb100074890 con 0x7fb1000b1480 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb10809b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb100074710 con 0x7fb100160350 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb10809b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb100074a10 con 0x7fb10015be30 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1000b1480 0x7fb1001f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1000b1480 0x7fb1001f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55660/0 (socket says 172.21.15.66:55660) 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 -- 172.21.15.66:0/3583600701 learned_addr learned my addr 172.21.15.66:0/3583600701 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105636640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb10015be30 0x7fb1000b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb106638640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb100160350 0x7fb1000c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 -- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb10015be30 msgr2=0x7fb1000b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb10015be30 0x7fb1000b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 -- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb100160350 msgr2=0x7fb1000c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb100160350 0x7fb1000c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105e37640 1 -- 172.21.15.66:0/3583600701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1001f6260 con 0x7fb1000b1480 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.543+0000 7fb105636640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb10015be30 0x7fb1000b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.544+0000 7fb105e37640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1000b1480 0x7fb1001f5d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009850 tx=0x7fb0f80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.544+0000 7fb0fe7fc640 1 -- 172.21.15.66:0/3583600701 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f803f070 con 0x7fb1000b1480 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.544+0000 7fb0fe7fc640 1 -- 172.21.15.66:0/3583600701 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0f8004890 con 0x7fb1000b1480 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.544+0000 7fb0fe7fc640 1 -- 172.21.15.66:0/3583600701 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8035030 con 0x7fb1000b1480 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.544+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1000c5050 con 0x7fb1000b1480 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.544+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1000c52f0 con 0x7fb1000b1480 2024-09-10T22:35:06.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.545+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb100074f30 con 0x7fb1000b1480 2024-09-10T22:35:06.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.545+0000 7fb0fe7fc640 1 -- 172.21.15.66:0/3583600701 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb0f8004ca0 con 0x7fb1000b1480 2024-09-10T22:35:06.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.546+0000 7fb0fe7fc640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0ec077f70 0x7fb0ec07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.546+0000 7fb0fe7fc640 1 -- 172.21.15.66:0/3583600701 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb0f80bb010 con 0x7fb1000b1480 2024-09-10T22:35:06.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.546+0000 7fb105636640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0ec077f70 0x7fb0ec07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.546+0000 7fb105636640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0ec077f70 0x7fb0ec07a430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1000ae050 tx=0x7fb0f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:06.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.550+0000 7fb0fe7fc640 1 -- 172.21.15.66:0/3583600701 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb0f8083b10 con 0x7fb1000b1480 2024-09-10T22:35:06.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.736+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb10006d810 con 0x7fb1000b1480 2024-09-10T22:35:06.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.737+0000 7fb0fe7fc640 1 -- 172.21.15.66:0/3583600701 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16641 (secure 0 0 0) 0x7fb0f8083410 con 0x7fb1000b1480 2024-09-10T22:35:06.738 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:06.738 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:04.052417+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:06.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0ec077f70 msgr2=0x7fb0ec07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0ec077f70 0x7fb0ec07a430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb1000ae050 tx=0x7fb0f4008040 comp rx=0 tx=0).stop 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1000b1480 msgr2=0x7fb1001f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1000b1480 0x7fb1001f5d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009850 tx=0x7fb0f80039e0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 shutdown_connections 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0ec077f70 0x7fb0ec07a430 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb100160350 0x7fb1000c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb10015be30 0x7fb1000b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 --2- 172.21.15.66:0/3583600701 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1000b1480 0x7fb1001f5d20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 >> 172.21.15.66:0/3583600701 conn(0x7fb1000d0430 msgr2=0x7fb1000c9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 shutdown_connections 2024-09-10T22:35:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.739+0000 7fb10809b640 1 -- 172.21.15.66:0/3583600701 wait complete. 2024-09-10T22:35:06.752 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:35:06.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 -- 172.21.15.66:0/4136143069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d2be0 msgr2=0x7f55340b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 --2- 172.21.15.66:0/4136143069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d2be0 0x7f55340b4450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5524005fd0 tx=0x7f552402aa90 comp rx=0 tx=0).stop 2024-09-10T22:35:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 -- 172.21.15.66:0/4136143069 shutdown_connections 2024-09-10T22:35:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 --2- 172.21.15.66:0/4136143069 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d2be0 0x7f55340b4450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 --2- 172.21.15.66:0/4136143069 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f553415e090 0x7f55340d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 -- 172.21.15.66:0/4136143069 >> 172.21.15.66:0/4136143069 conn(0x7f55340d3f10 msgr2=0x7f55340d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 -- 172.21.15.66:0/4136143069 shutdown_connections 2024-09-10T22:35:06.858 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.855+0000 7f5539184640 1 -- 172.21.15.66:0/4136143069 wait complete. 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 Processor -- start 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 -- start start 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d1ae0 0x7f553415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55340d2be0 0x7f553409dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553415e090 0x7f55340b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f55340b4d20 con 0x7f55340d1ae0 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55340b4ba0 con 0x7f553415e090 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5539184640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f55340b4ea0 con 0x7f55340d2be0 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.857+0000 7f5532d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d1ae0 0x7f553415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d1ae0 0x7f553415b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55674/0 (socket says 172.21.15.66:55674) 2024-09-10T22:35:06.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 -- 172.21.15.66:0/4262411468 learned_addr learned my addr 172.21.15.66:0/4262411468 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f553359e640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553415e090 0x7f55340b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f553259c640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55340d2be0 0x7f553409dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 -- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55340d2be0 msgr2=0x7f553409dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55340d2be0 0x7f553409dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 -- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553415e090 msgr2=0x7f55340b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553415e090 0x7f55340b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 -- 172.21.15.66:0/4262411468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55341f8280 con 0x7f55340d1ae0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5532d9d640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d1ae0 0x7f553415b9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d740 tx=0x7f552c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f552b7fe640 1 -- 172.21.15.66:0/4262411468 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c017070 con 0x7f55340d1ae0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f552b7fe640 1 -- 172.21.15.66:0/4262411468 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f552c009d20 con 0x7f55340d1ae0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f552b7fe640 1 -- 172.21.15.66:0/4262411468 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0106c0 con 0x7f55340d1ae0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341f84f0 con 0x7f55340d1ae0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.858+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341f8ab0 con 0x7f55340d1ae0 2024-09-10T22:35:06.862 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.859+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55340b4450 con 0x7f55340d1ae0 2024-09-10T22:35:06.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.860+0000 7f552b7fe640 1 -- 172.21.15.66:0/4262411468 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f552c010860 con 0x7f55340d1ae0 2024-09-10T22:35:06.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.860+0000 7f552b7fe640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5514077f70 0x7f551407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:06.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.860+0000 7f552b7fe640 1 -- 172.21.15.66:0/4262411468 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f552c09a200 con 0x7f55340d1ae0 2024-09-10T22:35:06.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.861+0000 7f553259c640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5514077f70 0x7f551407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:06.863 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.861+0000 7f553259c640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5514077f70 0x7f551407a430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f553406d370 tx=0x7f551c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:06.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:06.865+0000 7f552b7fe640 1 -- 172.21.15.66:0/4262411468 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f552c062d00 con 0x7f55340d1ae0 2024-09-10T22:35:07.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.044+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55340d2550 con 0x7f5514077f70 2024-09-10T22:35:07.050 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.049+0000 7f552b7fe640 1 -- 172.21.15.66:0/4262411468 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243982 (secure 0 0 0) 0x7f55340d2550 con 0x7f5514077f70 2024-09-10T22:35:07.053 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:07.054 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:07.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5514077f70 msgr2=0x7f551407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5514077f70 0x7f551407a430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f553406d370 tx=0x7f551c009040 comp rx=0 tx=0).stop 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d1ae0 msgr2=0x7f553415b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d1ae0 0x7f553415b9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f552c00d740 tx=0x7f552c00dc00 comp rx=0 tx=0).stop 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 shutdown_connections 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5514077f70 0x7f551407a430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f553415e090 0x7f55340b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f55340d2be0 0x7f553409dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 --2- 172.21.15.66:0/4262411468 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55340d1ae0 0x7f553415b9e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 >> 172.21.15.66:0/4262411468 conn(0x7f55340d3f10 msgr2=0x7f55340b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 shutdown_connections 2024-09-10T22:35:07.056 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.052+0000 7f5539184640 1 -- 172.21.15.66:0/4262411468 wait complete. 2024-09-10T22:35:07.065 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T22:35:06.593306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":122,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":2608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":2670592,"data_stored":1652692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.555133"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060843+0000","last_change":"2024-09-10T22:35:03.065907+0000","last_active":"2024-09-10T22:35:04.060843+0000","last_peered":"2024-09-10T22:35:04.060843+0000","last_clean":"2024-09-10T22:35:04.060843+0000","last_became_active":"2024-09-10T22:35:03.065607+0000","last_became_peered":"2024-09-10T22:35:03.065607+0000","last_unstale":"2024-09-10T22:35:04.060843+0000","last_undegraded":"2024-09-10T22:35:04.060843+0000","last_fullsized":"2024-09-10T22:35:04.060843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283932+0000","last_change":"2024-09-10T22:35:03.072149+0000","last_active":"2024-09-10T22:35:04.283932+0000","last_peered":"2024-09-10T22:35:04.283932+0000","last_clean":"2024-09-10T22:35:04.283932+0000","last_became_active":"2024-09-10T22:35:03.071819+0000","last_became_peered":"2024-09-10T22:35:03.071819+0000","last_unstale":"2024-09-10T22:35:04.283932+0000","last_undegraded":"2024-09-10T22:35:04.283932+0000","last_fullsized":"2024-09-10T22:35:04.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:26.466821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283542+0000","last_change":"2024-09-10T22:35:03.080808+0000","last_active":"2024-09-10T22:35:04.283542+0000","last_peered":"2024-09-10T22:35:04.283542+0000","last_clean":"2024-09-10T22:35:04.283542+0000","last_became_active":"2024-09-10T22:35:03.080467+0000","last_became_peered":"2024-09-10T22:35:03.080467+0000","last_unstale":"2024-09-10T22:35:04.283542+0000","last_undegraded":"2024-09-10T22:35:04.283542+0000","last_fullsized":"2024-09-10T22:35:04.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060665+0000","last_change":"2024-09-10T22:35:03.062056+0000","last_active":"2024-09-10T22:35:04.060665+0000","last_peered":"2024-09-10T22:35:04.060665+0000","last_clean":"2024-09-10T22:35:04.060665+0000","last_became_active":"2024-09-10T22:35:03.061769+0000","last_became_peered":"2024-09-10T22:35:03.061769+0000","last_unstale":"2024-09-10T22:35:04.060665+0000","last_undegraded":"2024-09-10T22:35:04.060665+0000","last_fullsized":"2024-09-10T22:35:04.060665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:02.406217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282700+0000","last_change":"2024-09-10T22:35:03.060118+0000","last_active":"2024-09-10T22:35:04.282700+0000","last_peered":"2024-09-10T22:35:04.282700+0000","last_clean":"2024-09-10T22:35:04.282700+0000","last_became_active":"2024-09-10T22:35:03.059799+0000","last_became_peered":"2024-09-10T22:35:03.059799+0000","last_unstale":"2024-09-10T22:35:04.282700+0000","last_undegraded":"2024-09-10T22:35:04.282700+0000","last_fullsized":"2024-09-10T22:35:04.282700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:50.965802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283468+0000","last_change":"2024-09-10T22:35:03.076617+0000","last_active":"2024-09-10T22:35:04.283468+0000","last_peered":"2024-09-10T22:35:04.283468+0000","last_clean":"2024-09-10T22:35:04.283468+0000","last_became_active":"2024-09-10T22:35:03.075527+0000","last_became_peered":"2024-09-10T22:35:03.075527+0000","last_unstale":"2024-09-10T22:35:04.283468+0000","last_undegraded":"2024-09-10T22:35:04.283468+0000","last_fullsized":"2024-09-10T22:35:04.283468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283390+0000","last_change":"2024-09-10T22:35:03.076662+0000","last_active":"2024-09-10T22:35:04.283390+0000","last_peered":"2024-09-10T22:35:04.283390+0000","last_clean":"2024-09-10T22:35:04.283390+0000","last_became_active":"2024-09-10T22:35:03.075696+0000","last_became_peered":"2024-09-10T22:35:03.075696+0000","last_unstale":"2024-09-10T22:35:04.283390+0000","last_undegraded":"2024-09-10T22:35:04.283390+0000","last_fullsized":"2024-09-10T22:35:04.283390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:59.070897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060768+0000","last_change":"2024-09-10T22:35:03.074633+0000","last_active":"2024-09-10T22:35:04.060768+0000","last_peered":"2024-09-10T22:35:04.060768+0000","last_clean":"2024-09-10T22:35:04.060768+0000","last_became_active":"2024-09-10T22:35:03.074332+0000","last_became_peered":"2024-09-10T22:35:03.074332+0000","last_unstale":"2024-09-10T22:35:04.060768+0000","last_undegraded":"2024-09-10T22:35:04.060768+0000","last_fullsized":"2024-09-10T22:35:04.060768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283311+0000","last_change":"2024-09-10T22:35:03.077344+0000","last_active":"2024-09-10T22:35:04.283311+0000","last_peered":"2024-09-10T22:35:04.283311+0000","last_clean":"2024-09-10T22:35:04.283311+0000","last_became_active":"2024-09-10T22:35:03.075668+0000","last_became_peered":"2024-09-10T22:35:03.075668+0000","last_unstale":"2024-09-10T22:35:04.283311+0000","last_undegraded":"2024-09-10T22:35:04.283311+0000","last_fullsized":"2024-09-10T22:35:04.283311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282910+0000","last_change":"2024-09-10T22:35:03.058708+0000","last_active":"2024-09-10T22:35:04.282910+0000","last_peered":"2024-09-10T22:35:04.282910+0000","last_clean":"2024-09-10T22:35:04.282910+0000","last_became_active":"2024-09-10T22:35:03.058399+0000","last_became_peered":"2024-09-10T22:35:03.058399+0000","last_unstale":"2024-09-10T22:35:04.282910+0000","last_undegraded":"2024-09-10T22:35:04.282910+0000","last_fullsized":"2024-09-10T22:35:04.282910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:21.971330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283873+0000","last_change":"2024-09-10T22:35:02.060438+0000","last_active":"2024-09-10T22:35:04.283873+0000","last_peered":"2024-09-10T22:35:04.283873+0000","last_clean":"2024-09-10T22:35:04.283873+0000","last_became_active":"2024-09-10T22:35:02.060124+0000","last_became_peered":"2024-09-10T22:35:02.060124+0000","last_unstale":"2024-09-10T22:35:04.283873+0000","last_undegraded":"2024-09-10T22:35:04.283873+0000","last_fullsized":"2024-09-10T22:35:04.283873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044841+0000","last_change":"2024-09-10T22:35:01.044841+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044841+0000","last_undegraded":"2024-09-10T22:35:01.044841+0000","last_fullsized":"2024-09-10T22:35:01.044841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283633+0000","last_change":"2024-09-10T22:35:02.058735+0000","last_active":"2024-09-10T22:35:04.283633+0000","last_peered":"2024-09-10T22:35:04.283633+0000","last_clean":"2024-09-10T22:35:04.283633+0000","last_became_active":"2024-09-10T22:35:02.058425+0000","last_became_peered":"2024-09-10T22:35:02.058425+0000","last_unstale":"2024-09-10T22:35:04.283633+0000","last_undegraded":"2024-09-10T22:35:04.283633+0000","last_fullsized":"2024-09-10T22:35:04.283633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284127+0000","last_change":"2024-09-10T22:35:03.070043+0000","last_active":"2024-09-10T22:35:04.284127+0000","last_peered":"2024-09-10T22:35:04.284127+0000","last_clean":"2024-09-10T22:35:04.284127+0000","last_became_active":"2024-09-10T22:35:03.069418+0000","last_became_peered":"2024-09-10T22:35:03.069418+0000","last_unstale":"2024-09-10T22:35:04.284127+0000","last_undegraded":"2024-09-10T22:35:04.284127+0000","last_fullsized":"2024-09-10T22:35:04.284127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:23.917793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.042994+0000","last_change":"2024-09-10T22:35:01.042994+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.042994+0000","last_undegraded":"2024-09-10T22:35:01.042994+0000","last_fullsized":"2024-09-10T22:35:01.042994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060647+0000","last_change":"2024-09-10T22:35:03.074156+0000","last_active":"2024-09-10T22:35:04.060647+0000","last_peered":"2024-09-10T22:35:04.060647+0000","last_clean":"2024-09-10T22:35:04.060647+0000","last_became_active":"2024-09-10T22:35:03.073836+0000","last_became_peered":"2024-09-10T22:35:03.073836+0000","last_unstale":"2024-09-10T22:35:04.060647+0000","last_undegraded":"2024-09-10T22:35:04.060647+0000","last_fullsized":"2024-09-10T22:35:04.060647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.045510+0000","last_change":"2024-09-10T22:35:01.045510+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.045510+0000","last_undegraded":"2024-09-10T22:35:01.045510+0000","last_fullsized":"2024-09-10T22:35:01.045510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043032+0000","last_change":"2024-09-10T22:35:01.043032+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043032+0000","last_undegraded":"2024-09-10T22:35:01.043032+0000","last_fullsized":"2024-09-10T22:35:01.043032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060881+0000","last_change":"2024-09-10T22:35:02.061366+0000","last_active":"2024-09-10T22:35:04.060881+0000","last_peered":"2024-09-10T22:35:04.060881+0000","last_clean":"2024-09-10T22:35:04.060881+0000","last_became_active":"2024-09-10T22:35:02.061092+0000","last_became_peered":"2024-09-10T22:35:02.061092+0000","last_unstale":"2024-09-10T22:35:04.060881+0000","last_undegraded":"2024-09-10T22:35:04.060881+0000","last_fullsized":"2024-09-10T22:35:04.060881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044803+0000","last_change":"2024-09-10T22:35:01.044803+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044803+0000","last_undegraded":"2024-09-10T22:35:01.044803+0000","last_fullsized":"2024-09-10T22:35:01.044803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043089+0000","last_change":"2024-09-10T22:35:01.043089+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043089+0000","last_undegraded":"2024-09-10T22:35:01.043089+0000","last_fullsized":"2024-09-10T22:35:01.043089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043732+0000","last_change":"2024-09-10T22:35:01.043732+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043732+0000","last_undegraded":"2024-09-10T22:35:01.043732+0000","last_fullsized":"2024-09-10T22:35:01.043732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044346+0000","last_change":"2024-09-10T22:35:01.044346+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044346+0000","last_undegraded":"2024-09-10T22:35:01.044346+0000","last_fullsized":"2024-09-10T22:35:01.044346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282601+0000","last_change":"2024-09-10T22:35:02.058914+0000","last_active":"2024-09-10T22:35:04.282601+0000","last_peered":"2024-09-10T22:35:04.282601+0000","last_clean":"2024-09-10T22:35:04.282601+0000","last_became_active":"2024-09-10T22:35:02.058054+0000","last_became_peered":"2024-09-10T22:35:02.058054+0000","last_unstale":"2024-09-10T22:35:04.282601+0000","last_undegraded":"2024-09-10T22:35:04.282601+0000","last_fullsized":"2024-09-10T22:35:04.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:07.958631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040146+0000","last_change":"2024-09-10T22:35:01.040146+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040146+0000","last_undegraded":"2024-09-10T22:35:01.040146+0000","last_fullsized":"2024-09-10T22:35:01.040146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282840+0000","last_change":"2024-09-10T22:35:02.056593+0000","last_active":"2024-09-10T22:35:04.282840+0000","last_peered":"2024-09-10T22:35:04.282840+0000","last_clean":"2024-09-10T22:35:04.282840+0000","last_became_active":"2024-09-10T22:35:02.056253+0000","last_became_peered":"2024-09-10T22:35:02.056253+0000","last_unstale":"2024-09-10T22:35:04.282840+0000","last_undegraded":"2024-09-10T22:35:04.282840+0000","last_fullsized":"2024-09-10T22:35:04.282840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060976+0000","last_change":"2024-09-10T22:35:02.059186+0000","last_active":"2024-09-10T22:35:04.060976+0000","last_peered":"2024-09-10T22:35:04.060976+0000","last_clean":"2024-09-10T22:35:04.060976+0000","last_became_active":"2024-09-10T22:35:02.058905+0000","last_became_peered":"2024-09-10T22:35:02.058905+0000","last_unstale":"2024-09-10T22:35:04.060976+0000","last_undegraded":"2024-09-10T22:35:04.060976+0000","last_fullsized":"2024-09-10T22:35:04.060976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283210+0000","last_change":"2024-09-10T22:35:02.059788+0000","last_active":"2024-09-10T22:35:04.283210+0000","last_peered":"2024-09-10T22:35:04.283210+0000","last_clean":"2024-09-10T22:35:04.283210+0000","last_became_active":"2024-09-10T22:35:02.059419+0000","last_became_peered":"2024-09-10T22:35:02.059419+0000","last_unstale":"2024-09-10T22:35:04.283210+0000","last_undegraded":"2024-09-10T22:35:04.283210+0000","last_fullsized":"2024-09-10T22:35:04.283210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T22:35:01.042201+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:01.042201+0000","last_peered":"2024-09-10T22:35:01.042201+0000","last_clean":"2024-09-10T22:35:01.042201+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:01.042201+0000","last_undegraded":"2024-09-10T22:35:01.042201+0000","last_fullsized":"2024-09-10T22:35:01.042201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043423+0000","last_change":"2024-09-10T22:35:01.043423+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043423+0000","last_undegraded":"2024-09-10T22:35:01.043423+0000","last_fullsized":"2024-09-10T22:35:01.043423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283212+0000","last_change":"2024-09-10T22:35:03.079281+0000","last_active":"2024-09-10T22:35:04.283212+0000","last_peered":"2024-09-10T22:35:04.283212+0000","last_clean":"2024-09-10T22:35:04.283212+0000","last_became_active":"2024-09-10T22:35:03.078916+0000","last_became_peered":"2024-09-10T22:35:03.078916+0000","last_unstale":"2024-09-10T22:35:04.283212+0000","last_undegraded":"2024-09-10T22:35:04.283212+0000","last_fullsized":"2024-09-10T22:35:04.283212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041127+0000","last_change":"2024-09-10T22:35:01.041127+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041127+0000","last_undegraded":"2024-09-10T22:35:01.041127+0000","last_fullsized":"2024-09-10T22:35:01.041127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283032+0000","last_change":"2024-09-10T22:35:02.057593+0000","last_active":"2024-09-10T22:35:04.283032+0000","last_peered":"2024-09-10T22:35:04.283032+0000","last_clean":"2024-09-10T22:35:04.283032+0000","last_became_active":"2024-09-10T22:35:02.057203+0000","last_became_peered":"2024-09-10T22:35:02.057203+0000","last_unstale":"2024-09-10T22:35:04.283032+0000","last_undegraded":"2024-09-10T22:35:04.283032+0000","last_fullsized":"2024-09-10T22:35:04.283032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:49.760894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060660+0000","last_change":"2024-09-10T22:35:03.057957+0000","last_active":"2024-09-10T22:35:04.060660+0000","last_peered":"2024-09-10T22:35:04.060660+0000","last_clean":"2024-09-10T22:35:04.060660+0000","last_became_active":"2024-09-10T22:35:03.057658+0000","last_became_peered":"2024-09-10T22:35:03.057658+0000","last_unstale":"2024-09-10T22:35:04.060660+0000","last_undegraded":"2024-09-10T22:35:04.060660+0000","last_fullsized":"2024-09-10T22:35:04.060660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043022+0000","last_change":"2024-09-10T22:35:01.043022+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043022+0000","last_undegraded":"2024-09-10T22:35:01.043022+0000","last_fullsized":"2024-09-10T22:35:01.043022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060864+0000","last_change":"2024-09-10T22:35:02.063524+0000","last_active":"2024-09-10T22:35:04.060864+0000","last_peered":"2024-09-10T22:35:04.060864+0000","last_clean":"2024-09-10T22:35:04.060864+0000","last_became_active":"2024-09-10T22:35:02.063232+0000","last_became_peered":"2024-09-10T22:35:02.063232+0000","last_unstale":"2024-09-10T22:35:04.060864+0000","last_undegraded":"2024-09-10T22:35:04.060864+0000","last_fullsized":"2024-09-10T22:35:04.060864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040400+0000","last_change":"2024-09-10T22:35:01.040400+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040400+0000","last_undegraded":"2024-09-10T22:35:01.040400+0000","last_fullsized":"2024-09-10T22:35:01.040400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060999+0000","last_change":"2024-09-10T22:35:03.058800+0000","last_active":"2024-09-10T22:35:04.060999+0000","last_peered":"2024-09-10T22:35:04.060999+0000","last_clean":"2024-09-10T22:35:04.060999+0000","last_became_active":"2024-09-10T22:35:03.058503+0000","last_became_peered":"2024-09-10T22:35:03.058503+0000","last_unstale":"2024-09-10T22:35:04.060999+0000","last_undegraded":"2024-09-10T22:35:04.060999+0000","last_fullsized":"2024-09-10T22:35:04.060999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.058112+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.057838+0000","last_became_peered":"2024-09-10T22:35:02.057838+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:20.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283672+0000","last_change":"2024-09-10T22:35:03.072054+0000","last_active":"2024-09-10T22:35:04.283672+0000","last_peered":"2024-09-10T22:35:04.283672+0000","last_clean":"2024-09-10T22:35:04.283672+0000","last_became_active":"2024-09-10T22:35:03.071739+0000","last_became_peered":"2024-09-10T22:35:03.071739+0000","last_unstale":"2024-09-10T22:35:04.283672+0000","last_undegraded":"2024-09-10T22:35:04.283672+0000","last_fullsized":"2024-09-10T22:35:04.283672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044311+0000","last_change":"2024-09-10T22:35:01.044311+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044311+0000","last_undegraded":"2024-09-10T22:35:01.044311+0000","last_fullsized":"2024-09-10T22:35:01.044311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041610+0000","last_change":"2024-09-10T22:35:01.041610+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041610+0000","last_undegraded":"2024-09-10T22:35:01.041610+0000","last_fullsized":"2024-09-10T22:35:01.041610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060764+0000","last_change":"2024-09-10T22:35:02.061273+0000","last_active":"2024-09-10T22:35:04.060764+0000","last_peered":"2024-09-10T22:35:04.060764+0000","last_clean":"2024-09-10T22:35:04.060764+0000","last_became_active":"2024-09-10T22:35:02.060953+0000","last_became_peered":"2024-09-10T22:35:02.060953+0000","last_unstale":"2024-09-10T22:35:04.060764+0000","last_undegraded":"2024-09-10T22:35:04.060764+0000","last_fullsized":"2024-09-10T22:35:04.060764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:32.172943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284309+0000","last_change":"2024-09-10T22:35:03.078424+0000","last_active":"2024-09-10T22:35:04.284309+0000","last_peered":"2024-09-10T22:35:04.284309+0000","last_clean":"2024-09-10T22:35:04.284309+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.284309+0000","last_undegraded":"2024-09-10T22:35:04.284309+0000","last_fullsized":"2024-09-10T22:35:04.284309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:48.985804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.061749+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.061248+0000","last_became_peered":"2024-09-10T22:35:02.061248+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:49.960770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283758+0000","last_change":"2024-09-10T22:35:03.076736+0000","last_active":"2024-09-10T22:35:04.283758+0000","last_peered":"2024-09-10T22:35:04.283758+0000","last_clean":"2024-09-10T22:35:04.283758+0000","last_became_active":"2024-09-10T22:35:03.075379+0000","last_became_peered":"2024-09-10T22:35:03.075379+0000","last_unstale":"2024-09-10T22:35:04.283758+0000","last_undegraded":"2024-09-10T22:35:04.283758+0000","last_fullsized":"2024-09-10T22:35:04.283758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.399411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040216+0000","last_change":"2024-09-10T22:35:01.040216+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040216+0000","last_undegraded":"2024-09-10T22:35:01.040216+0000","last_fullsized":"2024-09-10T22:35:01.040216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040170+0000","last_change":"2024-09-10T22:35:01.040170+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040170+0000","last_undegraded":"2024-09-10T22:35:01.040170+0000","last_fullsized":"2024-09-10T22:35:01.040170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041317+0000","last_change":"2024-09-10T22:35:01.041317+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041317+0000","last_undegraded":"2024-09-10T22:35:01.041317+0000","last_fullsized":"2024-09-10T22:35:01.041317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283509+0000","last_change":"2024-09-10T22:35:02.062827+0000","last_active":"2024-09-10T22:35:04.283509+0000","last_peered":"2024-09-10T22:35:04.283509+0000","last_clean":"2024-09-10T22:35:04.283509+0000","last_became_active":"2024-09-10T22:35:02.062342+0000","last_became_peered":"2024-09-10T22:35:02.062342+0000","last_unstale":"2024-09-10T22:35:04.283509+0000","last_undegraded":"2024-09-10T22:35:04.283509+0000","last_fullsized":"2024-09-10T22:35:04.283509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283072+0000","last_change":"2024-09-10T22:35:03.080815+0000","last_active":"2024-09-10T22:35:04.283072+0000","last_peered":"2024-09-10T22:35:04.283072+0000","last_clean":"2024-09-10T22:35:04.283072+0000","last_became_active":"2024-09-10T22:35:03.080484+0000","last_became_peered":"2024-09-10T22:35:03.080484+0000","last_unstale":"2024-09-10T22:35:04.283072+0000","last_undegraded":"2024-09-10T22:35:04.283072+0000","last_fullsized":"2024-09-10T22:35:04.283072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:58.700652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043161+0000","last_change":"2024-09-10T22:35:01.043161+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043161+0000","last_undegraded":"2024-09-10T22:35:01.043161+0000","last_fullsized":"2024-09-10T22:35:01.043161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:07.069 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.209+0000 7f84475b4640 1 -- 172.21.15.66:0/3684610587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c004df0 msgr2=0x7f842c005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.209+0000 7f84475b4640 1 --2- 172.21.15.66:0/3684610587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f842c004df0 0x7f842c005270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8438001ed0 tx=0x7f842c005830 comp rx=0 tx=0).stop 2024-09-10T22:35:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.209+0000 7f8445b51640 1 -- 172.21.15.66:0/3684610587 reap_dead start 2024-09-10T22:35:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.209+0000 7f84475b4640 1 -- 172.21.15.66:0/3684610587 shutdown_connections 2024-09-10T22:35:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.210+0000 7f84475b4640 1 -- 172.21.15.66:0/3684610587 >> 172.21.15.66:0/3684610587 conn(0x7f84400d1d20 msgr2=0x7f84400d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:07.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.210+0000 7f84475b4640 1 -- 172.21.15.66:0/3684610587 shutdown_connections 2024-09-10T22:35:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.210+0000 7f84475b4640 1 -- 172.21.15.66:0/3684610587 wait complete. 2024-09-10T22:35:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.211+0000 7f84475b4640 1 Processor -- start 2024-09-10T22:35:07.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.211+0000 7f84475b4640 1 -- start start 2024-09-10T22:35:07.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f84475b4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c004df0 0x7f84401f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f84475b4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8440158860 0x7f8440150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f84475b4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84400b18e0 0x7f84400b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f84475b4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f844015f7f0 con 0x7f84400b18e0 2024-09-10T22:35:07.214 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f84475b4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f844015f670 con 0x7f8440158860 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f84475b4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f84400ce3c0 con 0x7f842c004df0 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84400b18e0 0x7f84400b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84400b18e0 0x7f84400b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55696/0 (socket says 172.21.15.66:55696) 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 -- 172.21.15.66:0/1569690764 learned_addr learned my addr 172.21.15.66:0/1569690764 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445350640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c004df0 0x7f84401f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8444b4f640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8440158860 0x7f8440150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 -- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c004df0 msgr2=0x7f84401f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c004df0 0x7f84401f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 -- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8440158860 msgr2=0x7f8440150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8440158860 0x7f8440150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 -- 172.21.15.66:0/1569690764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84400b4da0 con 0x7f84400b18e0 2024-09-10T22:35:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8445b51640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84400b18e0 0x7f84400b1d70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8438001ff0 tx=0x7f8438006810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8435ffb640 1 -- 172.21.15.66:0/1569690764 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438001af0 con 0x7f84400b18e0 2024-09-10T22:35:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8435ffb640 1 -- 172.21.15.66:0/1569690764 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8438003d40 con 0x7f84400b18e0 2024-09-10T22:35:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84400b5080 con 0x7f84400b18e0 2024-09-10T22:35:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.212+0000 7f8435ffb640 1 -- 172.21.15.66:0/1569690764 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438006c40 con 0x7f84400b18e0 2024-09-10T22:35:07.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.213+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84400b5640 con 0x7f84400b18e0 2024-09-10T22:35:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.214+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84400cd830 con 0x7f84400b18e0 2024-09-10T22:35:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.214+0000 7f8435ffb640 1 -- 172.21.15.66:0/1569690764 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8438047030 con 0x7f84400b18e0 2024-09-10T22:35:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.215+0000 7f8435ffb640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8424077f70 0x7f842407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.215+0000 7f8435ffb640 1 -- 172.21.15.66:0/1569690764 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f84380b9860 con 0x7f84400b18e0 2024-09-10T22:35:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.215+0000 7f8445350640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8424077f70 0x7f842407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.215+0000 7f8445350640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8424077f70 0x7f842407a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f843c001880 tx=0x7f843c001550 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:07.220 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.219+0000 7f8435ffb640 1 -- 172.21.15.66:0/1569690764 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8438082360 con 0x7f84400b18e0 2024-09-10T22:35:07.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.400+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84400b4b50 con 0x7f8424077f70 2024-09-10T22:35:07.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.404+0000 7f8435ffb640 1 -- 172.21.15.66:0/1569690764 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243982 (secure 0 0 0) 0x7f84400b4b50 con 0x7f8424077f70 2024-09-10T22:35:07.407 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:07.408 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8424077f70 msgr2=0x7f842407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8424077f70 0x7f842407a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f843c001880 tx=0x7f843c001550 comp rx=0 tx=0).stop 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84400b18e0 msgr2=0x7f84400b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84400b18e0 0x7f84400b1d70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8438001ff0 tx=0x7f8438006810 comp rx=0 tx=0).stop 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 shutdown_connections 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8424077f70 0x7f842407a430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84400b18e0 0x7f84400b1d70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8440158860 0x7f8440150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 --2- 172.21.15.66:0/1569690764 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f842c004df0 0x7f84401f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 >> 172.21.15.66:0/1569690764 conn(0x7f84400d1d20 msgr2=0x7f84400aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 shutdown_connections 2024-09-10T22:35:07.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.407+0000 7f84475b4640 1 -- 172.21.15.66:0/1569690764 wait complete. 2024-09-10T22:35:07.419 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T22:35:06.593306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":122,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":2608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":2670592,"data_stored":1652692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.555133"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060843+0000","last_change":"2024-09-10T22:35:03.065907+0000","last_active":"2024-09-10T22:35:04.060843+0000","last_peered":"2024-09-10T22:35:04.060843+0000","last_clean":"2024-09-10T22:35:04.060843+0000","last_became_active":"2024-09-10T22:35:03.065607+0000","last_became_peered":"2024-09-10T22:35:03.065607+0000","last_unstale":"2024-09-10T22:35:04.060843+0000","last_undegraded":"2024-09-10T22:35:04.060843+0000","last_fullsized":"2024-09-10T22:35:04.060843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283932+0000","last_change":"2024-09-10T22:35:03.072149+0000","last_active":"2024-09-10T22:35:04.283932+0000","last_peered":"2024-09-10T22:35:04.283932+0000","last_clean":"2024-09-10T22:35:04.283932+0000","last_became_active":"2024-09-10T22:35:03.071819+0000","last_became_peered":"2024-09-10T22:35:03.071819+0000","last_unstale":"2024-09-10T22:35:04.283932+0000","last_undegraded":"2024-09-10T22:35:04.283932+0000","last_fullsized":"2024-09-10T22:35:04.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:26.466821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283542+0000","last_change":"2024-09-10T22:35:03.080808+0000","last_active":"2024-09-10T22:35:04.283542+0000","last_peered":"2024-09-10T22:35:04.283542+0000","last_clean":"2024-09-10T22:35:04.283542+0000","last_became_active":"2024-09-10T22:35:03.080467+0000","last_became_peered":"2024-09-10T22:35:03.080467+0000","last_unstale":"2024-09-10T22:35:04.283542+0000","last_undegraded":"2024-09-10T22:35:04.283542+0000","last_fullsized":"2024-09-10T22:35:04.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060665+0000","last_change":"2024-09-10T22:35:03.062056+0000","last_active":"2024-09-10T22:35:04.060665+0000","last_peered":"2024-09-10T22:35:04.060665+0000","last_clean":"2024-09-10T22:35:04.060665+0000","last_became_active":"2024-09-10T22:35:03.061769+0000","last_became_peered":"2024-09-10T22:35:03.061769+0000","last_unstale":"2024-09-10T22:35:04.060665+0000","last_undegraded":"2024-09-10T22:35:04.060665+0000","last_fullsized":"2024-09-10T22:35:04.060665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:02.406217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282700+0000","last_change":"2024-09-10T22:35:03.060118+0000","last_active":"2024-09-10T22:35:04.282700+0000","last_peered":"2024-09-10T22:35:04.282700+0000","last_clean":"2024-09-10T22:35:04.282700+0000","last_became_active":"2024-09-10T22:35:03.059799+0000","last_became_peered":"2024-09-10T22:35:03.059799+0000","last_unstale":"2024-09-10T22:35:04.282700+0000","last_undegraded":"2024-09-10T22:35:04.282700+0000","last_fullsized":"2024-09-10T22:35:04.282700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:50.965802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283468+0000","last_change":"2024-09-10T22:35:03.076617+0000","last_active":"2024-09-10T22:35:04.283468+0000","last_peered":"2024-09-10T22:35:04.283468+0000","last_clean":"2024-09-10T22:35:04.283468+0000","last_became_active":"2024-09-10T22:35:03.075527+0000","last_became_peered":"2024-09-10T22:35:03.075527+0000","last_unstale":"2024-09-10T22:35:04.283468+0000","last_undegraded":"2024-09-10T22:35:04.283468+0000","last_fullsized":"2024-09-10T22:35:04.283468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283390+0000","last_change":"2024-09-10T22:35:03.076662+0000","last_active":"2024-09-10T22:35:04.283390+0000","last_peered":"2024-09-10T22:35:04.283390+0000","last_clean":"2024-09-10T22:35:04.283390+0000","last_became_active":"2024-09-10T22:35:03.075696+0000","last_became_peered":"2024-09-10T22:35:03.075696+0000","last_unstale":"2024-09-10T22:35:04.283390+0000","last_undegraded":"2024-09-10T22:35:04.283390+0000","last_fullsized":"2024-09-10T22:35:04.283390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:59.070897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060768+0000","last_change":"2024-09-10T22:35:03.074633+0000","last_active":"2024-09-10T22:35:04.060768+0000","last_peered":"2024-09-10T22:35:04.060768+0000","last_clean":"2024-09-10T22:35:04.060768+0000","last_became_active":"2024-09-10T22:35:03.074332+0000","last_became_peered":"2024-09-10T22:35:03.074332+0000","last_unstale":"2024-09-10T22:35:04.060768+0000","last_undegraded":"2024-09-10T22:35:04.060768+0000","last_fullsized":"2024-09-10T22:35:04.060768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283311+0000","last_change":"2024-09-10T22:35:03.077344+0000","last_active":"2024-09-10T22:35:04.283311+0000","last_peered":"2024-09-10T22:35:04.283311+0000","last_clean":"2024-09-10T22:35:04.283311+0000","last_became_active":"2024-09-10T22:35:03.075668+0000","last_became_peered":"2024-09-10T22:35:03.075668+0000","last_unstale":"2024-09-10T22:35:04.283311+0000","last_undegraded":"2024-09-10T22:35:04.283311+0000","last_fullsized":"2024-09-10T22:35:04.283311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282910+0000","last_change":"2024-09-10T22:35:03.058708+0000","last_active":"2024-09-10T22:35:04.282910+0000","last_peered":"2024-09-10T22:35:04.282910+0000","last_clean":"2024-09-10T22:35:04.282910+0000","last_became_active":"2024-09-10T22:35:03.058399+0000","last_became_peered":"2024-09-10T22:35:03.058399+0000","last_unstale":"2024-09-10T22:35:04.282910+0000","last_undegraded":"2024-09-10T22:35:04.282910+0000","last_fullsized":"2024-09-10T22:35:04.282910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:21.971330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283873+0000","last_change":"2024-09-10T22:35:02.060438+0000","last_active":"2024-09-10T22:35:04.283873+0000","last_peered":"2024-09-10T22:35:04.283873+0000","last_clean":"2024-09-10T22:35:04.283873+0000","last_became_active":"2024-09-10T22:35:02.060124+0000","last_became_peered":"2024-09-10T22:35:02.060124+0000","last_unstale":"2024-09-10T22:35:04.283873+0000","last_undegraded":"2024-09-10T22:35:04.283873+0000","last_fullsized":"2024-09-10T22:35:04.283873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044841+0000","last_change":"2024-09-10T22:35:01.044841+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044841+0000","last_undegraded":"2024-09-10T22:35:01.044841+0000","last_fullsized":"2024-09-10T22:35:01.044841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283633+0000","last_change":"2024-09-10T22:35:02.058735+0000","last_active":"2024-09-10T22:35:04.283633+0000","last_peered":"2024-09-10T22:35:04.283633+0000","last_clean":"2024-09-10T22:35:04.283633+0000","last_became_active":"2024-09-10T22:35:02.058425+0000","last_became_peered":"2024-09-10T22:35:02.058425+0000","last_unstale":"2024-09-10T22:35:04.283633+0000","last_undegraded":"2024-09-10T22:35:04.283633+0000","last_fullsized":"2024-09-10T22:35:04.283633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284127+0000","last_change":"2024-09-10T22:35:03.070043+0000","last_active":"2024-09-10T22:35:04.284127+0000","last_peered":"2024-09-10T22:35:04.284127+0000","last_clean":"2024-09-10T22:35:04.284127+0000","last_became_active":"2024-09-10T22:35:03.069418+0000","last_became_peered":"2024-09-10T22:35:03.069418+0000","last_unstale":"2024-09-10T22:35:04.284127+0000","last_undegraded":"2024-09-10T22:35:04.284127+0000","last_fullsized":"2024-09-10T22:35:04.284127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:23.917793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.042994+0000","last_change":"2024-09-10T22:35:01.042994+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.042994+0000","last_undegraded":"2024-09-10T22:35:01.042994+0000","last_fullsized":"2024-09-10T22:35:01.042994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060647+0000","last_change":"2024-09-10T22:35:03.074156+0000","last_active":"2024-09-10T22:35:04.060647+0000","last_peered":"2024-09-10T22:35:04.060647+0000","last_clean":"2024-09-10T22:35:04.060647+0000","last_became_active":"2024-09-10T22:35:03.073836+0000","last_became_peered":"2024-09-10T22:35:03.073836+0000","last_unstale":"2024-09-10T22:35:04.060647+0000","last_undegraded":"2024-09-10T22:35:04.060647+0000","last_fullsized":"2024-09-10T22:35:04.060647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.045510+0000","last_change":"2024-09-10T22:35:01.045510+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.045510+0000","last_undegraded":"2024-09-10T22:35:01.045510+0000","last_fullsized":"2024-09-10T22:35:01.045510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043032+0000","last_change":"2024-09-10T22:35:01.043032+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043032+0000","last_undegraded":"2024-09-10T22:35:01.043032+0000","last_fullsized":"2024-09-10T22:35:01.043032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060881+0000","last_change":"2024-09-10T22:35:02.061366+0000","last_active":"2024-09-10T22:35:04.060881+0000","last_peered":"2024-09-10T22:35:04.060881+0000","last_clean":"2024-09-10T22:35:04.060881+0000","last_became_active":"2024-09-10T22:35:02.061092+0000","last_became_peered":"2024-09-10T22:35:02.061092+0000","last_unstale":"2024-09-10T22:35:04.060881+0000","last_undegraded":"2024-09-10T22:35:04.060881+0000","last_fullsized":"2024-09-10T22:35:04.060881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044803+0000","last_change":"2024-09-10T22:35:01.044803+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044803+0000","last_undegraded":"2024-09-10T22:35:01.044803+0000","last_fullsized":"2024-09-10T22:35:01.044803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043089+0000","last_change":"2024-09-10T22:35:01.043089+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043089+0000","last_undegraded":"2024-09-10T22:35:01.043089+0000","last_fullsized":"2024-09-10T22:35:01.043089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043732+0000","last_change":"2024-09-10T22:35:01.043732+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043732+0000","last_undegraded":"2024-09-10T22:35:01.043732+0000","last_fullsized":"2024-09-10T22:35:01.043732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044346+0000","last_change":"2024-09-10T22:35:01.044346+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044346+0000","last_undegraded":"2024-09-10T22:35:01.044346+0000","last_fullsized":"2024-09-10T22:35:01.044346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282601+0000","last_change":"2024-09-10T22:35:02.058914+0000","last_active":"2024-09-10T22:35:04.282601+0000","last_peered":"2024-09-10T22:35:04.282601+0000","last_clean":"2024-09-10T22:35:04.282601+0000","last_became_active":"2024-09-10T22:35:02.058054+0000","last_became_peered":"2024-09-10T22:35:02.058054+0000","last_unstale":"2024-09-10T22:35:04.282601+0000","last_undegraded":"2024-09-10T22:35:04.282601+0000","last_fullsized":"2024-09-10T22:35:04.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:07.958631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040146+0000","last_change":"2024-09-10T22:35:01.040146+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040146+0000","last_undegraded":"2024-09-10T22:35:01.040146+0000","last_fullsized":"2024-09-10T22:35:01.040146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282840+0000","last_change":"2024-09-10T22:35:02.056593+0000","last_active":"2024-09-10T22:35:04.282840+0000","last_peered":"2024-09-10T22:35:04.282840+0000","last_clean":"2024-09-10T22:35:04.282840+0000","last_became_active":"2024-09-10T22:35:02.056253+0000","last_became_peered":"2024-09-10T22:35:02.056253+0000","last_unstale":"2024-09-10T22:35:04.282840+0000","last_undegraded":"2024-09-10T22:35:04.282840+0000","last_fullsized":"2024-09-10T22:35:04.282840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060976+0000","last_change":"2024-09-10T22:35:02.059186+0000","last_active":"2024-09-10T22:35:04.060976+0000","last_peered":"2024-09-10T22:35:04.060976+0000","last_clean":"2024-09-10T22:35:04.060976+0000","last_became_active":"2024-09-10T22:35:02.058905+0000","last_became_peered":"2024-09-10T22:35:02.058905+0000","last_unstale":"2024-09-10T22:35:04.060976+0000","last_undegraded":"2024-09-10T22:35:04.060976+0000","last_fullsized":"2024-09-10T22:35:04.060976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283210+0000","last_change":"2024-09-10T22:35:02.059788+0000","last_active":"2024-09-10T22:35:04.283210+0000","last_peered":"2024-09-10T22:35:04.283210+0000","last_clean":"2024-09-10T22:35:04.283210+0000","last_became_active":"2024-09-10T22:35:02.059419+0000","last_became_peered":"2024-09-10T22:35:02.059419+0000","last_unstale":"2024-09-10T22:35:04.283210+0000","last_undegraded":"2024-09-10T22:35:04.283210+0000","last_fullsized":"2024-09-10T22:35:04.283210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T22:35:01.042201+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:01.042201+0000","last_peered":"2024-09-10T22:35:01.042201+0000","last_clean":"2024-09-10T22:35:01.042201+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:01.042201+0000","last_undegraded":"2024-09-10T22:35:01.042201+0000","last_fullsized":"2024-09-10T22:35:01.042201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043423+0000","last_change":"2024-09-10T22:35:01.043423+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043423+0000","last_undegraded":"2024-09-10T22:35:01.043423+0000","last_fullsized":"2024-09-10T22:35:01.043423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283212+0000","last_change":"2024-09-10T22:35:03.079281+0000","last_active":"2024-09-10T22:35:04.283212+0000","last_peered":"2024-09-10T22:35:04.283212+0000","last_clean":"2024-09-10T22:35:04.283212+0000","last_became_active":"2024-09-10T22:35:03.078916+0000","last_became_peered":"2024-09-10T22:35:03.078916+0000","last_unstale":"2024-09-10T22:35:04.283212+0000","last_undegraded":"2024-09-10T22:35:04.283212+0000","last_fullsized":"2024-09-10T22:35:04.283212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041127+0000","last_change":"2024-09-10T22:35:01.041127+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041127+0000","last_undegraded":"2024-09-10T22:35:01.041127+0000","last_fullsized":"2024-09-10T22:35:01.041127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283032+0000","last_change":"2024-09-10T22:35:02.057593+0000","last_active":"2024-09-10T22:35:04.283032+0000","last_peered":"2024-09-10T22:35:04.283032+0000","last_clean":"2024-09-10T22:35:04.283032+0000","last_became_active":"2024-09-10T22:35:02.057203+0000","last_became_peered":"2024-09-10T22:35:02.057203+0000","last_unstale":"2024-09-10T22:35:04.283032+0000","last_undegraded":"2024-09-10T22:35:04.283032+0000","last_fullsized":"2024-09-10T22:35:04.283032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:49.760894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060660+0000","last_change":"2024-09-10T22:35:03.057957+0000","last_active":"2024-09-10T22:35:04.060660+0000","last_peered":"2024-09-10T22:35:04.060660+0000","last_clean":"2024-09-10T22:35:04.060660+0000","last_became_active":"2024-09-10T22:35:03.057658+0000","last_became_peered":"2024-09-10T22:35:03.057658+0000","last_unstale":"2024-09-10T22:35:04.060660+0000","last_undegraded":"2024-09-10T22:35:04.060660+0000","last_fullsized":"2024-09-10T22:35:04.060660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043022+0000","last_change":"2024-09-10T22:35:01.043022+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043022+0000","last_undegraded":"2024-09-10T22:35:01.043022+0000","last_fullsized":"2024-09-10T22:35:01.043022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060864+0000","last_change":"2024-09-10T22:35:02.063524+0000","last_active":"2024-09-10T22:35:04.060864+0000","last_peered":"2024-09-10T22:35:04.060864+0000","last_clean":"2024-09-10T22:35:04.060864+0000","last_became_active":"2024-09-10T22:35:02.063232+0000","last_became_peered":"2024-09-10T22:35:02.063232+0000","last_unstale":"2024-09-10T22:35:04.060864+0000","last_undegraded":"2024-09-10T22:35:04.060864+0000","last_fullsized":"2024-09-10T22:35:04.060864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040400+0000","last_change":"2024-09-10T22:35:01.040400+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040400+0000","last_undegraded":"2024-09-10T22:35:01.040400+0000","last_fullsized":"2024-09-10T22:35:01.040400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060999+0000","last_change":"2024-09-10T22:35:03.058800+0000","last_active":"2024-09-10T22:35:04.060999+0000","last_peered":"2024-09-10T22:35:04.060999+0000","last_clean":"2024-09-10T22:35:04.060999+0000","last_became_active":"2024-09-10T22:35:03.058503+0000","last_became_peered":"2024-09-10T22:35:03.058503+0000","last_unstale":"2024-09-10T22:35:04.060999+0000","last_undegraded":"2024-09-10T22:35:04.060999+0000","last_fullsized":"2024-09-10T22:35:04.060999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.058112+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.057838+0000","last_became_peered":"2024-09-10T22:35:02.057838+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:20.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283672+0000","last_change":"2024-09-10T22:35:03.072054+0000","last_active":"2024-09-10T22:35:04.283672+0000","last_peered":"2024-09-10T22:35:04.283672+0000","last_clean":"2024-09-10T22:35:04.283672+0000","last_became_active":"2024-09-10T22:35:03.071739+0000","last_became_peered":"2024-09-10T22:35:03.071739+0000","last_unstale":"2024-09-10T22:35:04.283672+0000","last_undegraded":"2024-09-10T22:35:04.283672+0000","last_fullsized":"2024-09-10T22:35:04.283672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044311+0000","last_change":"2024-09-10T22:35:01.044311+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044311+0000","last_undegraded":"2024-09-10T22:35:01.044311+0000","last_fullsized":"2024-09-10T22:35:01.044311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041610+0000","last_change":"2024-09-10T22:35:01.041610+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041610+0000","last_undegraded":"2024-09-10T22:35:01.041610+0000","last_fullsized":"2024-09-10T22:35:01.041610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060764+0000","last_change":"2024-09-10T22:35:02.061273+0000","last_active":"2024-09-10T22:35:04.060764+0000","last_peered":"2024-09-10T22:35:04.060764+0000","last_clean":"2024-09-10T22:35:04.060764+0000","last_became_active":"2024-09-10T22:35:02.060953+0000","last_became_peered":"2024-09-10T22:35:02.060953+0000","last_unstale":"2024-09-10T22:35:04.060764+0000","last_undegraded":"2024-09-10T22:35:04.060764+0000","last_fullsized":"2024-09-10T22:35:04.060764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:32.172943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284309+0000","last_change":"2024-09-10T22:35:03.078424+0000","last_active":"2024-09-10T22:35:04.284309+0000","last_peered":"2024-09-10T22:35:04.284309+0000","last_clean":"2024-09-10T22:35:04.284309+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.284309+0000","last_undegraded":"2024-09-10T22:35:04.284309+0000","last_fullsized":"2024-09-10T22:35:04.284309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:48.985804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.061749+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.061248+0000","last_became_peered":"2024-09-10T22:35:02.061248+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:49.960770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283758+0000","last_change":"2024-09-10T22:35:03.076736+0000","last_active":"2024-09-10T22:35:04.283758+0000","last_peered":"2024-09-10T22:35:04.283758+0000","last_clean":"2024-09-10T22:35:04.283758+0000","last_became_active":"2024-09-10T22:35:03.075379+0000","last_became_peered":"2024-09-10T22:35:03.075379+0000","last_unstale":"2024-09-10T22:35:04.283758+0000","last_undegraded":"2024-09-10T22:35:04.283758+0000","last_fullsized":"2024-09-10T22:35:04.283758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.399411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040216+0000","last_change":"2024-09-10T22:35:01.040216+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040216+0000","last_undegraded":"2024-09-10T22:35:01.040216+0000","last_fullsized":"2024-09-10T22:35:01.040216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040170+0000","last_change":"2024-09-10T22:35:01.040170+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040170+0000","last_undegraded":"2024-09-10T22:35:01.040170+0000","last_fullsized":"2024-09-10T22:35:01.040170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041317+0000","last_change":"2024-09-10T22:35:01.041317+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041317+0000","last_undegraded":"2024-09-10T22:35:01.041317+0000","last_fullsized":"2024-09-10T22:35:01.041317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283509+0000","last_change":"2024-09-10T22:35:02.062827+0000","last_active":"2024-09-10T22:35:04.283509+0000","last_peered":"2024-09-10T22:35:04.283509+0000","last_clean":"2024-09-10T22:35:04.283509+0000","last_became_active":"2024-09-10T22:35:02.062342+0000","last_became_peered":"2024-09-10T22:35:02.062342+0000","last_unstale":"2024-09-10T22:35:04.283509+0000","last_undegraded":"2024-09-10T22:35:04.283509+0000","last_fullsized":"2024-09-10T22:35:04.283509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283072+0000","last_change":"2024-09-10T22:35:03.080815+0000","last_active":"2024-09-10T22:35:04.283072+0000","last_peered":"2024-09-10T22:35:04.283072+0000","last_clean":"2024-09-10T22:35:04.283072+0000","last_became_active":"2024-09-10T22:35:03.080484+0000","last_became_peered":"2024-09-10T22:35:03.080484+0000","last_unstale":"2024-09-10T22:35:04.283072+0000","last_undegraded":"2024-09-10T22:35:04.283072+0000","last_fullsized":"2024-09-10T22:35:04.283072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:58.700652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043161+0000","last_change":"2024-09-10T22:35:01.043161+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043161+0000","last_undegraded":"2024-09-10T22:35:01.043161+0000","last_fullsized":"2024-09-10T22:35:01.043161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:07.423 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:07.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 -- 172.21.15.66:0/1230511790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 msgr2=0x7f6154074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 --2- 172.21.15.66:0/1230511790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f6154074710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6154072680 tx=0x7f614402a4d0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 -- 172.21.15.66:0/1230511790 shutdown_connections 2024-09-10T22:35:07.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 --2- 172.21.15.66:0/1230511790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f6154074710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 --2- 172.21.15.66:0/1230511790 >> v2:172.21.15.94:3300/0 conn(0x7f615415e4c0 0x7f61540741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 -- 172.21.15.66:0/1230511790 >> 172.21.15.66:0/1230511790 conn(0x7f61540d0430 msgr2=0x7f61540d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:07.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 -- 172.21.15.66:0/1230511790 shutdown_connections 2024-09-10T22:35:07.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.564+0000 7f615c1ec640 1 -- 172.21.15.66:0/1230511790 wait complete. 2024-09-10T22:35:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.565+0000 7f615c1ec640 1 Processor -- start 2024-09-10T22:35:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.565+0000 7f615c1ec640 1 -- start start 2024-09-10T22:35:07.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f615c1ec640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61540af570 0x7f61540c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f615c1ec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f61540c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f615c1ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415e4c0 0x7f61540b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f615c1ec640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f61540b0c50 con 0x7f61540cca30 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f615c1ec640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61540b0ad0 con 0x7f615415e4c0 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f615c1ec640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f615415f1d0 con 0x7f61540af570 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f61540c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f61540c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55710/0 (socket says 172.21.15.66:55710) 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 -- 172.21.15.66:0/2081475492 learned_addr learned my addr 172.21.15.66:0/2081475492 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159f88640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61540af570 0x7f61540c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f615a789640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415e4c0 0x7f61540b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 -- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61540af570 msgr2=0x7f61540c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61540af570 0x7f61540c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 -- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415e4c0 msgr2=0x7f61540b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415e4c0 0x7f61540b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.566+0000 7f6159787640 1 -- 172.21.15.66:0/2081475492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61540c3f60 con 0x7f61540cca30 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.567+0000 7f6159787640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f61540c6080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6144009850 tx=0x7f61440039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.567+0000 7f614a7fc640 1 -- 172.21.15.66:0/2081475492 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614403f070 con 0x7f61540cca30 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.567+0000 7f614a7fc640 1 -- 172.21.15.66:0/2081475492 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6144004890 con 0x7f61540cca30 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.567+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61540c4240 con 0x7f61540cca30 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.567+0000 7f614a7fc640 1 -- 172.21.15.66:0/2081475492 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144035030 con 0x7f61540cca30 2024-09-10T22:35:07.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.567+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61540c4800 con 0x7f61540cca30 2024-09-10T22:35:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.568+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61540c4fd0 con 0x7f61540cca30 2024-09-10T22:35:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.569+0000 7f614a7fc640 1 -- 172.21.15.66:0/2081475492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6144004ca0 con 0x7f61540cca30 2024-09-10T22:35:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.569+0000 7f614a7fc640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6138077f70 0x7f613807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.569+0000 7f614a7fc640 1 -- 172.21.15.66:0/2081475492 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f61440baf50 con 0x7f61540cca30 2024-09-10T22:35:07.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.569+0000 7f6159f88640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6138077f70 0x7f613807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.570+0000 7f6159f88640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6138077f70 0x7f613807a430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61500058c0 tx=0x7f6150005f60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:07.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.573+0000 7f614a7fc640 1 -- 172.21.15.66:0/2081475492 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6154074f20 con 0x7f61540cca30 2024-09-10T22:35:07.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.756+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61541f4180 con 0x7f6138077f70 2024-09-10T22:35:07.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.760+0000 7f614a7fc640 1 -- 172.21.15.66:0/2081475492 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243982 (secure 0 0 0) 0x7f61541f4180 con 0x7f6138077f70 2024-09-10T22:35:07.764 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:07.765 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:07.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6138077f70 msgr2=0x7f613807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6138077f70 0x7f613807a430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61500058c0 tx=0x7f6150005f60 comp rx=0 tx=0).stop 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 msgr2=0x7f61540c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f61540c6080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6144009850 tx=0x7f61440039e0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 shutdown_connections 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6138077f70 0x7f613807a430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f615415e4c0 0x7f61540b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61540cca30 0x7f61540c6080 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 --2- 172.21.15.66:0/2081475492 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f61540af570 0x7f61540c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 >> 172.21.15.66:0/2081475492 conn(0x7f61540d0430 msgr2=0x7f615406ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 shutdown_connections 2024-09-10T22:35:07.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.763+0000 7f615c1ec640 1 -- 172.21.15.66:0/2081475492 wait complete. 2024-09-10T22:35:07.775 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T22:35:06.593306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":122,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":2608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":2670592,"data_stored":1652692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.555133"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060843+0000","last_change":"2024-09-10T22:35:03.065907+0000","last_active":"2024-09-10T22:35:04.060843+0000","last_peered":"2024-09-10T22:35:04.060843+0000","last_clean":"2024-09-10T22:35:04.060843+0000","last_became_active":"2024-09-10T22:35:03.065607+0000","last_became_peered":"2024-09-10T22:35:03.065607+0000","last_unstale":"2024-09-10T22:35:04.060843+0000","last_undegraded":"2024-09-10T22:35:04.060843+0000","last_fullsized":"2024-09-10T22:35:04.060843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283932+0000","last_change":"2024-09-10T22:35:03.072149+0000","last_active":"2024-09-10T22:35:04.283932+0000","last_peered":"2024-09-10T22:35:04.283932+0000","last_clean":"2024-09-10T22:35:04.283932+0000","last_became_active":"2024-09-10T22:35:03.071819+0000","last_became_peered":"2024-09-10T22:35:03.071819+0000","last_unstale":"2024-09-10T22:35:04.283932+0000","last_undegraded":"2024-09-10T22:35:04.283932+0000","last_fullsized":"2024-09-10T22:35:04.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:26.466821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283542+0000","last_change":"2024-09-10T22:35:03.080808+0000","last_active":"2024-09-10T22:35:04.283542+0000","last_peered":"2024-09-10T22:35:04.283542+0000","last_clean":"2024-09-10T22:35:04.283542+0000","last_became_active":"2024-09-10T22:35:03.080467+0000","last_became_peered":"2024-09-10T22:35:03.080467+0000","last_unstale":"2024-09-10T22:35:04.283542+0000","last_undegraded":"2024-09-10T22:35:04.283542+0000","last_fullsized":"2024-09-10T22:35:04.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060665+0000","last_change":"2024-09-10T22:35:03.062056+0000","last_active":"2024-09-10T22:35:04.060665+0000","last_peered":"2024-09-10T22:35:04.060665+0000","last_clean":"2024-09-10T22:35:04.060665+0000","last_became_active":"2024-09-10T22:35:03.061769+0000","last_became_peered":"2024-09-10T22:35:03.061769+0000","last_unstale":"2024-09-10T22:35:04.060665+0000","last_undegraded":"2024-09-10T22:35:04.060665+0000","last_fullsized":"2024-09-10T22:35:04.060665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:02.406217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282700+0000","last_change":"2024-09-10T22:35:03.060118+0000","last_active":"2024-09-10T22:35:04.282700+0000","last_peered":"2024-09-10T22:35:04.282700+0000","last_clean":"2024-09-10T22:35:04.282700+0000","last_became_active":"2024-09-10T22:35:03.059799+0000","last_became_peered":"2024-09-10T22:35:03.059799+0000","last_unstale":"2024-09-10T22:35:04.282700+0000","last_undegraded":"2024-09-10T22:35:04.282700+0000","last_fullsized":"2024-09-10T22:35:04.282700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:50.965802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283468+0000","last_change":"2024-09-10T22:35:03.076617+0000","last_active":"2024-09-10T22:35:04.283468+0000","last_peered":"2024-09-10T22:35:04.283468+0000","last_clean":"2024-09-10T22:35:04.283468+0000","last_became_active":"2024-09-10T22:35:03.075527+0000","last_became_peered":"2024-09-10T22:35:03.075527+0000","last_unstale":"2024-09-10T22:35:04.283468+0000","last_undegraded":"2024-09-10T22:35:04.283468+0000","last_fullsized":"2024-09-10T22:35:04.283468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283390+0000","last_change":"2024-09-10T22:35:03.076662+0000","last_active":"2024-09-10T22:35:04.283390+0000","last_peered":"2024-09-10T22:35:04.283390+0000","last_clean":"2024-09-10T22:35:04.283390+0000","last_became_active":"2024-09-10T22:35:03.075696+0000","last_became_peered":"2024-09-10T22:35:03.075696+0000","last_unstale":"2024-09-10T22:35:04.283390+0000","last_undegraded":"2024-09-10T22:35:04.283390+0000","last_fullsized":"2024-09-10T22:35:04.283390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:59.070897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060768+0000","last_change":"2024-09-10T22:35:03.074633+0000","last_active":"2024-09-10T22:35:04.060768+0000","last_peered":"2024-09-10T22:35:04.060768+0000","last_clean":"2024-09-10T22:35:04.060768+0000","last_became_active":"2024-09-10T22:35:03.074332+0000","last_became_peered":"2024-09-10T22:35:03.074332+0000","last_unstale":"2024-09-10T22:35:04.060768+0000","last_undegraded":"2024-09-10T22:35:04.060768+0000","last_fullsized":"2024-09-10T22:35:04.060768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283311+0000","last_change":"2024-09-10T22:35:03.077344+0000","last_active":"2024-09-10T22:35:04.283311+0000","last_peered":"2024-09-10T22:35:04.283311+0000","last_clean":"2024-09-10T22:35:04.283311+0000","last_became_active":"2024-09-10T22:35:03.075668+0000","last_became_peered":"2024-09-10T22:35:03.075668+0000","last_unstale":"2024-09-10T22:35:04.283311+0000","last_undegraded":"2024-09-10T22:35:04.283311+0000","last_fullsized":"2024-09-10T22:35:04.283311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282910+0000","last_change":"2024-09-10T22:35:03.058708+0000","last_active":"2024-09-10T22:35:04.282910+0000","last_peered":"2024-09-10T22:35:04.282910+0000","last_clean":"2024-09-10T22:35:04.282910+0000","last_became_active":"2024-09-10T22:35:03.058399+0000","last_became_peered":"2024-09-10T22:35:03.058399+0000","last_unstale":"2024-09-10T22:35:04.282910+0000","last_undegraded":"2024-09-10T22:35:04.282910+0000","last_fullsized":"2024-09-10T22:35:04.282910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:21.971330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283873+0000","last_change":"2024-09-10T22:35:02.060438+0000","last_active":"2024-09-10T22:35:04.283873+0000","last_peered":"2024-09-10T22:35:04.283873+0000","last_clean":"2024-09-10T22:35:04.283873+0000","last_became_active":"2024-09-10T22:35:02.060124+0000","last_became_peered":"2024-09-10T22:35:02.060124+0000","last_unstale":"2024-09-10T22:35:04.283873+0000","last_undegraded":"2024-09-10T22:35:04.283873+0000","last_fullsized":"2024-09-10T22:35:04.283873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044841+0000","last_change":"2024-09-10T22:35:01.044841+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044841+0000","last_undegraded":"2024-09-10T22:35:01.044841+0000","last_fullsized":"2024-09-10T22:35:01.044841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283633+0000","last_change":"2024-09-10T22:35:02.058735+0000","last_active":"2024-09-10T22:35:04.283633+0000","last_peered":"2024-09-10T22:35:04.283633+0000","last_clean":"2024-09-10T22:35:04.283633+0000","last_became_active":"2024-09-10T22:35:02.058425+0000","last_became_peered":"2024-09-10T22:35:02.058425+0000","last_unstale":"2024-09-10T22:35:04.283633+0000","last_undegraded":"2024-09-10T22:35:04.283633+0000","last_fullsized":"2024-09-10T22:35:04.283633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284127+0000","last_change":"2024-09-10T22:35:03.070043+0000","last_active":"2024-09-10T22:35:04.284127+0000","last_peered":"2024-09-10T22:35:04.284127+0000","last_clean":"2024-09-10T22:35:04.284127+0000","last_became_active":"2024-09-10T22:35:03.069418+0000","last_became_peered":"2024-09-10T22:35:03.069418+0000","last_unstale":"2024-09-10T22:35:04.284127+0000","last_undegraded":"2024-09-10T22:35:04.284127+0000","last_fullsized":"2024-09-10T22:35:04.284127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:23.917793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.042994+0000","last_change":"2024-09-10T22:35:01.042994+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.042994+0000","last_undegraded":"2024-09-10T22:35:01.042994+0000","last_fullsized":"2024-09-10T22:35:01.042994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060647+0000","last_change":"2024-09-10T22:35:03.074156+0000","last_active":"2024-09-10T22:35:04.060647+0000","last_peered":"2024-09-10T22:35:04.060647+0000","last_clean":"2024-09-10T22:35:04.060647+0000","last_became_active":"2024-09-10T22:35:03.073836+0000","last_became_peered":"2024-09-10T22:35:03.073836+0000","last_unstale":"2024-09-10T22:35:04.060647+0000","last_undegraded":"2024-09-10T22:35:04.060647+0000","last_fullsized":"2024-09-10T22:35:04.060647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.045510+0000","last_change":"2024-09-10T22:35:01.045510+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.045510+0000","last_undegraded":"2024-09-10T22:35:01.045510+0000","last_fullsized":"2024-09-10T22:35:01.045510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043032+0000","last_change":"2024-09-10T22:35:01.043032+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043032+0000","last_undegraded":"2024-09-10T22:35:01.043032+0000","last_fullsized":"2024-09-10T22:35:01.043032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060881+0000","last_change":"2024-09-10T22:35:02.061366+0000","last_active":"2024-09-10T22:35:04.060881+0000","last_peered":"2024-09-10T22:35:04.060881+0000","last_clean":"2024-09-10T22:35:04.060881+0000","last_became_active":"2024-09-10T22:35:02.061092+0000","last_became_peered":"2024-09-10T22:35:02.061092+0000","last_unstale":"2024-09-10T22:35:04.060881+0000","last_undegraded":"2024-09-10T22:35:04.060881+0000","last_fullsized":"2024-09-10T22:35:04.060881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044803+0000","last_change":"2024-09-10T22:35:01.044803+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044803+0000","last_undegraded":"2024-09-10T22:35:01.044803+0000","last_fullsized":"2024-09-10T22:35:01.044803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043089+0000","last_change":"2024-09-10T22:35:01.043089+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043089+0000","last_undegraded":"2024-09-10T22:35:01.043089+0000","last_fullsized":"2024-09-10T22:35:01.043089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043732+0000","last_change":"2024-09-10T22:35:01.043732+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043732+0000","last_undegraded":"2024-09-10T22:35:01.043732+0000","last_fullsized":"2024-09-10T22:35:01.043732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044346+0000","last_change":"2024-09-10T22:35:01.044346+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044346+0000","last_undegraded":"2024-09-10T22:35:01.044346+0000","last_fullsized":"2024-09-10T22:35:01.044346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282601+0000","last_change":"2024-09-10T22:35:02.058914+0000","last_active":"2024-09-10T22:35:04.282601+0000","last_peered":"2024-09-10T22:35:04.282601+0000","last_clean":"2024-09-10T22:35:04.282601+0000","last_became_active":"2024-09-10T22:35:02.058054+0000","last_became_peered":"2024-09-10T22:35:02.058054+0000","last_unstale":"2024-09-10T22:35:04.282601+0000","last_undegraded":"2024-09-10T22:35:04.282601+0000","last_fullsized":"2024-09-10T22:35:04.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:07.958631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040146+0000","last_change":"2024-09-10T22:35:01.040146+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040146+0000","last_undegraded":"2024-09-10T22:35:01.040146+0000","last_fullsized":"2024-09-10T22:35:01.040146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282840+0000","last_change":"2024-09-10T22:35:02.056593+0000","last_active":"2024-09-10T22:35:04.282840+0000","last_peered":"2024-09-10T22:35:04.282840+0000","last_clean":"2024-09-10T22:35:04.282840+0000","last_became_active":"2024-09-10T22:35:02.056253+0000","last_became_peered":"2024-09-10T22:35:02.056253+0000","last_unstale":"2024-09-10T22:35:04.282840+0000","last_undegraded":"2024-09-10T22:35:04.282840+0000","last_fullsized":"2024-09-10T22:35:04.282840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060976+0000","last_change":"2024-09-10T22:35:02.059186+0000","last_active":"2024-09-10T22:35:04.060976+0000","last_peered":"2024-09-10T22:35:04.060976+0000","last_clean":"2024-09-10T22:35:04.060976+0000","last_became_active":"2024-09-10T22:35:02.058905+0000","last_became_peered":"2024-09-10T22:35:02.058905+0000","last_unstale":"2024-09-10T22:35:04.060976+0000","last_undegraded":"2024-09-10T22:35:04.060976+0000","last_fullsized":"2024-09-10T22:35:04.060976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283210+0000","last_change":"2024-09-10T22:35:02.059788+0000","last_active":"2024-09-10T22:35:04.283210+0000","last_peered":"2024-09-10T22:35:04.283210+0000","last_clean":"2024-09-10T22:35:04.283210+0000","last_became_active":"2024-09-10T22:35:02.059419+0000","last_became_peered":"2024-09-10T22:35:02.059419+0000","last_unstale":"2024-09-10T22:35:04.283210+0000","last_undegraded":"2024-09-10T22:35:04.283210+0000","last_fullsized":"2024-09-10T22:35:04.283210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T22:35:01.042201+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:01.042201+0000","last_peered":"2024-09-10T22:35:01.042201+0000","last_clean":"2024-09-10T22:35:01.042201+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:01.042201+0000","last_undegraded":"2024-09-10T22:35:01.042201+0000","last_fullsized":"2024-09-10T22:35:01.042201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043423+0000","last_change":"2024-09-10T22:35:01.043423+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043423+0000","last_undegraded":"2024-09-10T22:35:01.043423+0000","last_fullsized":"2024-09-10T22:35:01.043423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283212+0000","last_change":"2024-09-10T22:35:03.079281+0000","last_active":"2024-09-10T22:35:04.283212+0000","last_peered":"2024-09-10T22:35:04.283212+0000","last_clean":"2024-09-10T22:35:04.283212+0000","last_became_active":"2024-09-10T22:35:03.078916+0000","last_became_peered":"2024-09-10T22:35:03.078916+0000","last_unstale":"2024-09-10T22:35:04.283212+0000","last_undegraded":"2024-09-10T22:35:04.283212+0000","last_fullsized":"2024-09-10T22:35:04.283212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041127+0000","last_change":"2024-09-10T22:35:01.041127+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041127+0000","last_undegraded":"2024-09-10T22:35:01.041127+0000","last_fullsized":"2024-09-10T22:35:01.041127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283032+0000","last_change":"2024-09-10T22:35:02.057593+0000","last_active":"2024-09-10T22:35:04.283032+0000","last_peered":"2024-09-10T22:35:04.283032+0000","last_clean":"2024-09-10T22:35:04.283032+0000","last_became_active":"2024-09-10T22:35:02.057203+0000","last_became_peered":"2024-09-10T22:35:02.057203+0000","last_unstale":"2024-09-10T22:35:04.283032+0000","last_undegraded":"2024-09-10T22:35:04.283032+0000","last_fullsized":"2024-09-10T22:35:04.283032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:49.760894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060660+0000","last_change":"2024-09-10T22:35:03.057957+0000","last_active":"2024-09-10T22:35:04.060660+0000","last_peered":"2024-09-10T22:35:04.060660+0000","last_clean":"2024-09-10T22:35:04.060660+0000","last_became_active":"2024-09-10T22:35:03.057658+0000","last_became_peered":"2024-09-10T22:35:03.057658+0000","last_unstale":"2024-09-10T22:35:04.060660+0000","last_undegraded":"2024-09-10T22:35:04.060660+0000","last_fullsized":"2024-09-10T22:35:04.060660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043022+0000","last_change":"2024-09-10T22:35:01.043022+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043022+0000","last_undegraded":"2024-09-10T22:35:01.043022+0000","last_fullsized":"2024-09-10T22:35:01.043022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060864+0000","last_change":"2024-09-10T22:35:02.063524+0000","last_active":"2024-09-10T22:35:04.060864+0000","last_peered":"2024-09-10T22:35:04.060864+0000","last_clean":"2024-09-10T22:35:04.060864+0000","last_became_active":"2024-09-10T22:35:02.063232+0000","last_became_peered":"2024-09-10T22:35:02.063232+0000","last_unstale":"2024-09-10T22:35:04.060864+0000","last_undegraded":"2024-09-10T22:35:04.060864+0000","last_fullsized":"2024-09-10T22:35:04.060864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040400+0000","last_change":"2024-09-10T22:35:01.040400+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040400+0000","last_undegraded":"2024-09-10T22:35:01.040400+0000","last_fullsized":"2024-09-10T22:35:01.040400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060999+0000","last_change":"2024-09-10T22:35:03.058800+0000","last_active":"2024-09-10T22:35:04.060999+0000","last_peered":"2024-09-10T22:35:04.060999+0000","last_clean":"2024-09-10T22:35:04.060999+0000","last_became_active":"2024-09-10T22:35:03.058503+0000","last_became_peered":"2024-09-10T22:35:03.058503+0000","last_unstale":"2024-09-10T22:35:04.060999+0000","last_undegraded":"2024-09-10T22:35:04.060999+0000","last_fullsized":"2024-09-10T22:35:04.060999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.058112+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.057838+0000","last_became_peered":"2024-09-10T22:35:02.057838+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:20.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283672+0000","last_change":"2024-09-10T22:35:03.072054+0000","last_active":"2024-09-10T22:35:04.283672+0000","last_peered":"2024-09-10T22:35:04.283672+0000","last_clean":"2024-09-10T22:35:04.283672+0000","last_became_active":"2024-09-10T22:35:03.071739+0000","last_became_peered":"2024-09-10T22:35:03.071739+0000","last_unstale":"2024-09-10T22:35:04.283672+0000","last_undegraded":"2024-09-10T22:35:04.283672+0000","last_fullsized":"2024-09-10T22:35:04.283672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044311+0000","last_change":"2024-09-10T22:35:01.044311+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044311+0000","last_undegraded":"2024-09-10T22:35:01.044311+0000","last_fullsized":"2024-09-10T22:35:01.044311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041610+0000","last_change":"2024-09-10T22:35:01.041610+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041610+0000","last_undegraded":"2024-09-10T22:35:01.041610+0000","last_fullsized":"2024-09-10T22:35:01.041610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060764+0000","last_change":"2024-09-10T22:35:02.061273+0000","last_active":"2024-09-10T22:35:04.060764+0000","last_peered":"2024-09-10T22:35:04.060764+0000","last_clean":"2024-09-10T22:35:04.060764+0000","last_became_active":"2024-09-10T22:35:02.060953+0000","last_became_peered":"2024-09-10T22:35:02.060953+0000","last_unstale":"2024-09-10T22:35:04.060764+0000","last_undegraded":"2024-09-10T22:35:04.060764+0000","last_fullsized":"2024-09-10T22:35:04.060764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:32.172943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284309+0000","last_change":"2024-09-10T22:35:03.078424+0000","last_active":"2024-09-10T22:35:04.284309+0000","last_peered":"2024-09-10T22:35:04.284309+0000","last_clean":"2024-09-10T22:35:04.284309+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.284309+0000","last_undegraded":"2024-09-10T22:35:04.284309+0000","last_fullsized":"2024-09-10T22:35:04.284309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:48.985804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.061749+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.061248+0000","last_became_peered":"2024-09-10T22:35:02.061248+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:49.960770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283758+0000","last_change":"2024-09-10T22:35:03.076736+0000","last_active":"2024-09-10T22:35:04.283758+0000","last_peered":"2024-09-10T22:35:04.283758+0000","last_clean":"2024-09-10T22:35:04.283758+0000","last_became_active":"2024-09-10T22:35:03.075379+0000","last_became_peered":"2024-09-10T22:35:03.075379+0000","last_unstale":"2024-09-10T22:35:04.283758+0000","last_undegraded":"2024-09-10T22:35:04.283758+0000","last_fullsized":"2024-09-10T22:35:04.283758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.399411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040216+0000","last_change":"2024-09-10T22:35:01.040216+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040216+0000","last_undegraded":"2024-09-10T22:35:01.040216+0000","last_fullsized":"2024-09-10T22:35:01.040216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040170+0000","last_change":"2024-09-10T22:35:01.040170+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040170+0000","last_undegraded":"2024-09-10T22:35:01.040170+0000","last_fullsized":"2024-09-10T22:35:01.040170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041317+0000","last_change":"2024-09-10T22:35:01.041317+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041317+0000","last_undegraded":"2024-09-10T22:35:01.041317+0000","last_fullsized":"2024-09-10T22:35:01.041317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283509+0000","last_change":"2024-09-10T22:35:02.062827+0000","last_active":"2024-09-10T22:35:04.283509+0000","last_peered":"2024-09-10T22:35:04.283509+0000","last_clean":"2024-09-10T22:35:04.283509+0000","last_became_active":"2024-09-10T22:35:02.062342+0000","last_became_peered":"2024-09-10T22:35:02.062342+0000","last_unstale":"2024-09-10T22:35:04.283509+0000","last_undegraded":"2024-09-10T22:35:04.283509+0000","last_fullsized":"2024-09-10T22:35:04.283509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283072+0000","last_change":"2024-09-10T22:35:03.080815+0000","last_active":"2024-09-10T22:35:04.283072+0000","last_peered":"2024-09-10T22:35:04.283072+0000","last_clean":"2024-09-10T22:35:04.283072+0000","last_became_active":"2024-09-10T22:35:03.080484+0000","last_became_peered":"2024-09-10T22:35:03.080484+0000","last_unstale":"2024-09-10T22:35:04.283072+0000","last_undegraded":"2024-09-10T22:35:04.283072+0000","last_fullsized":"2024-09-10T22:35:04.283072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:58.700652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043161+0000","last_change":"2024-09-10T22:35:01.043161+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043161+0000","last_undegraded":"2024-09-10T22:35:01.043161+0000","last_fullsized":"2024-09-10T22:35:01.043161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:07.778 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-10T22:35:07.778 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.779 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-10T22:35:07.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-10T22:35:07.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-10T22:35:07.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.044841+0000', 'last_change': '2024-09-10T22:35:01.044841+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.044841+0000', 'last_undegraded': '2024-09-10T22:35:01.044841+0000', 'last_fullsized': '2024-09-10T22:35:01.044841+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.042994+0000', 'last_change': '2024-09-10T22:35:01.042994+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.042994+0000', 'last_undegraded': '2024-09-10T22:35:01.042994+0000', 'last_fullsized': '2024-09-10T22:35:01.042994+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.045510+0000', 'last_change': '2024-09-10T22:35:01.045510+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.045510+0000', 'last_undegraded': '2024-09-10T22:35:01.045510+0000', 'last_fullsized': '2024-09-10T22:35:01.045510+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-10T22:35:07.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.043032+0000', 'last_change': '2024-09-10T22:35:01.043032+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.043032+0000', 'last_undegraded': '2024-09-10T22:35:01.043032+0000', 'last_fullsized': '2024-09-10T22:35:01.043032+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.044803+0000', 'last_change': '2024-09-10T22:35:01.044803+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.044803+0000', 'last_undegraded': '2024-09-10T22:35:01.044803+0000', 'last_fullsized': '2024-09-10T22:35:01.044803+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.043089+0000', 'last_change': '2024-09-10T22:35:01.043089+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.043089+0000', 'last_undegraded': '2024-09-10T22:35:01.043089+0000', 'last_fullsized': '2024-09-10T22:35:01.043089+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T22:35:07.784 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.043732+0000', 'last_change': '2024-09-10T22:35:01.043732+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.043732+0000', 'last_undegraded': '2024-09-10T22:35:01.043732+0000', 'last_fullsized': '2024-09-10T22:35:01.043732+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.044346+0000', 'last_change': '2024-09-10T22:35:01.044346+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.044346+0000', 'last_undegraded': '2024-09-10T22:35:01.044346+0000', 'last_fullsized': '2024-09-10T22:35:01.044346+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-10T22:35:07.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.040146+0000', 'last_change': '2024-09-10T22:35:01.040146+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.040146+0000', 'last_undegraded': '2024-09-10T22:35:01.040146+0000', 'last_fullsized': '2024-09-10T22:35:01.040146+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-10T22:35:07.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.043423+0000', 'last_change': '2024-09-10T22:35:01.043423+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.043423+0000', 'last_undegraded': '2024-09-10T22:35:01.043423+0000', 'last_fullsized': '2024-09-10T22:35:01.043423+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.041127+0000', 'last_change': '2024-09-10T22:35:01.041127+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.041127+0000', 'last_undegraded': '2024-09-10T22:35:01.041127+0000', 'last_fullsized': '2024-09-10T22:35:01.041127+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.043022+0000', 'last_change': '2024-09-10T22:35:01.043022+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.043022+0000', 'last_undegraded': '2024-09-10T22:35:01.043022+0000', 'last_fullsized': '2024-09-10T22:35:01.043022+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-10T22:35:07.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.040400+0000', 'last_change': '2024-09-10T22:35:01.040400+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.040400+0000', 'last_undegraded': '2024-09-10T22:35:01.040400+0000', 'last_fullsized': '2024-09-10T22:35:01.040400+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.044311+0000', 'last_change': '2024-09-10T22:35:01.044311+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.044311+0000', 'last_undegraded': '2024-09-10T22:35:01.044311+0000', 'last_fullsized': '2024-09-10T22:35:01.044311+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.041610+0000', 'last_change': '2024-09-10T22:35:01.041610+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.041610+0000', 'last_undegraded': '2024-09-10T22:35:01.041610+0000', 'last_fullsized': '2024-09-10T22:35:01.041610+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.040216+0000', 'last_change': '2024-09-10T22:35:01.040216+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.040216+0000', 'last_undegraded': '2024-09-10T22:35:01.040216+0000', 'last_fullsized': '2024-09-10T22:35:01.040216+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:35:07.788 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.040170+0000', 'last_change': '2024-09-10T22:35:01.040170+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.040170+0000', 'last_undegraded': '2024-09-10T22:35:01.040170+0000', 'last_fullsized': '2024-09-10T22:35:01.040170+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.041317+0000', 'last_change': '2024-09-10T22:35:01.041317+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.041317+0000', 'last_undegraded': '2024-09-10T22:35:01.041317+0000', 'last_fullsized': '2024-09-10T22:35:01.041317+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T22:35:02.037934+0000', 'last_change': '2024-09-10T22:35:02.037934+0000', 'last_active': '2024-09-10T22:35:02.037934+0000', 'last_peered': '2024-09-10T22:35:02.037934+0000', 'last_clean': '2024-09-10T22:35:02.037934+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:02.037934+0000', 'last_undegraded': '2024-09-10T22:35:02.037934+0000', 'last_fullsized': '2024-09-10T22:35:02.037934+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:02.037934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T22:35:07.789 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-10T22:35:07.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T22:35:01.043161+0000', 'last_change': '2024-09-10T22:35:01.043161+0000', 'last_active': '2024-09-10T22:35:01.032902+0000', 'last_peered': '2024-09-10T22:35:01.032902+0000', 'last_clean': '2024-09-10T22:35:01.032902+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T22:35:01.043161+0000', 'last_undegraded': '2024-09-10T22:35:01.043161+0000', 'last_fullsized': '2024-09-10T22:35:01.043161+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'last_clean_scrub_stamp': '2024-09-10T22:35:01.032902+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T22:35:07.790 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3891151889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 msgr2=0x7fdbf40d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3891151889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40d1180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009830 tx=0x7fdbec02a510 comp rx=0 tx=0).stop 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3891151889 shutdown_connections 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3891151889 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf40d1ae0 0x7fdbf40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3891151889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40d1180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3891151889 >> 172.21.15.66:0/3891151889 conn(0x7fdbf40d3f10 msgr2=0x7fdbf40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3891151889 shutdown_connections 2024-09-10T22:35:07.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.918+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3891151889 wait complete. 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.919+0000 7fdbfbcef640 1 Processor -- start 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfbcef640 1 -- start start 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfbcef640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf40d1ae0 0x7fdbf41f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfbcef640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf40d2a10 0x7fdbf414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfbcef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfbcef640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf40cf1c0 con 0x7fdbf415e090 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfbcef640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf40b5670 con 0x7fdbf40d2a10 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfbcef640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf40cf340 con 0x7fdbf40d1ae0 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfa28c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfa28c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55724/0 (socket says 172.21.15.66:55724) 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbfa28c640 1 -- 172.21.15.66:0/3191157748 learned_addr learned my addr 172.21.15.66:0/3191157748 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:07.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.920+0000 7fdbf9a8b640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf40d1ae0 0x7fdbf41f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbf928a640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf40d2a10 0x7fdbf414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfa28c640 1 -- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf40d1ae0 msgr2=0x7fdbf41f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfa28c640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf40d1ae0 0x7fdbf41f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfa28c640 1 -- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf40d2a10 msgr2=0x7fdbf414ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfa28c640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf40d2a10 0x7fdbf414ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfa28c640 1 -- 172.21.15.66:0/3191157748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf41f52e0 con 0x7fdbf415e090 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfa28c640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40a0730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400c3a0 tx=0x7fdbe400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbf27fc640 1 -- 172.21.15.66:0/3191157748 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4017070 con 0x7fdbf415e090 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbf27fc640 1 -- 172.21.15.66:0/3191157748 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbe40035f0 con 0x7fdbf415e090 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf4151a40 con 0x7fdbf415e090 2024-09-10T22:35:07.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbf27fc640 1 -- 172.21.15.66:0/3191157748 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4003a40 con 0x7fdbf415e090 2024-09-10T22:35:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.921+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf4152000 con 0x7fdbf415e090 2024-09-10T22:35:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.922+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf40cf860 con 0x7fdbf415e090 2024-09-10T22:35:07.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.923+0000 7fdbf27fc640 1 -- 172.21.15.66:0/3191157748 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdbe4003be0 con 0x7fdbf415e090 2024-09-10T22:35:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.923+0000 7fdbf27fc640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdbe0077f70 0x7fdbe007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.923+0000 7fdbf27fc640 1 -- 172.21.15.66:0/3191157748 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdbe4099200 con 0x7fdbf415e090 2024-09-10T22:35:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.923+0000 7fdbf9a8b640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdbe0077f70 0x7fdbe007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:07.926 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.924+0000 7fdbf9a8b640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdbe0077f70 0x7fdbe007a430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdbec034000 tx=0x7fdbec004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:07.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:07.928+0000 7fdbf27fc640 1 -- 172.21.15.66:0/3191157748 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fdbe4061740 con 0x7fdbf415e090 2024-09-10T22:35:08.028 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:35:08.027+0000 7f12bfdea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:35:08.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.187+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fdbf40b6420 con 0x7fdbf415e090 2024-09-10T22:35:08.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.187+0000 7fdbf27fc640 1 -- 172.21.15.66:0/3191157748 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1214 (secure 0 0 0) 0x7fdbe4066170 con 0x7fdbf415e090 2024-09-10T22:35:08.189 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:08.189 INFO:teuthology.orchestra.run.smithi066.stdout:{"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1726007690,"num_in_osds":8,"osd_in_since":1726007681,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":45},{"state_name":"active+clean","count":33},{"state_name":"creating+peering","count":19}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":590528,"bytes_used":241631232,"bytes_avail":772852482048,"bytes_total":773094113280,"unknown_pgs_ratio":0.46391752362251282,"inactive_pgs_ratio":0.19587628543376923,"write_bytes_sec":368,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"c","status":"up:active","gid":4380}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T22:34:38.228612+0000","services":{}},"progress_events":{}} 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdbe0077f70 msgr2=0x7fdbe007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdbe0077f70 0x7fdbe007a430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdbec034000 tx=0x7fdbec004730 comp rx=0 tx=0).stop 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 msgr2=0x7fdbf40a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40a0730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400c3a0 tx=0x7fdbe400c860 comp rx=0 tx=0).stop 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 shutdown_connections 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdbe0077f70 0x7fdbe007a430 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdbf415e090 0x7fdbf40a0730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdbf40d2a10 0x7fdbf414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 --2- 172.21.15.66:0/3191157748 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdbf40d1ae0 0x7fdbf41f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 >> 172.21.15.66:0/3191157748 conn(0x7fdbf40d3f10 msgr2=0x7fdbf40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 shutdown_connections 2024-09-10T22:35:08.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.190+0000 7fdbfbcef640 1 -- 172.21.15.66:0/3191157748 wait complete. 2024-09-10T22:35:08.201 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T22:35:08.202 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:08.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 -- 172.21.15.66:0/2857332843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 msgr2=0x7f172c0d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:08.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 --2- 172.21.15.66:0/2857332843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0d0950 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c02a5d0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 -- 172.21.15.66:0/2857332843 shutdown_connections 2024-09-10T22:35:08.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 --2- 172.21.15.66:0/2857332843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0d0950 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 --2- 172.21.15.66:0/2857332843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f172c161640 0x7f172c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 -- 172.21.15.66:0/2857332843 >> 172.21.15.66:0/2857332843 conn(0x7f172c0d1d20 msgr2=0x7f172c0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:08.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 -- 172.21.15.66:0/2857332843 shutdown_connections 2024-09-10T22:35:08.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.306+0000 7f17337f3640 1 -- 172.21.15.66:0/2857332843 wait complete. 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.307+0000 7f17337f3640 1 Processor -- start 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.307+0000 7f17337f3640 1 -- start start 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f17337f3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f17337f3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f172c0ce6a0 0x7f172c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f17337f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c161640 0x7f172c1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f17337f3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f172c06e200 con 0x7f172c0cdc00 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f17337f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f172c06e080 con 0x7f172c161640 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f17337f3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f172c06e380 con 0x7f172c0ce6a0 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55748/0 (socket says 172.21.15.66:55748) 2024-09-10T22:35:08.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 -- 172.21.15.66:0/352677036 learned_addr learned my addr 172.21.15.66:0/352677036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f1730d8e640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f172c0ce6a0 0x7f172c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f1731d90640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c161640 0x7f172c1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 -- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f172c0ce6a0 msgr2=0x7f172c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f172c0ce6a0 0x7f172c15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 -- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c161640 msgr2=0x7f172c1601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c161640 0x7f172c1601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 -- 172.21.15.66:0/352677036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c0c9eb0 con 0x7f172c0cdc00 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f1731d90640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c161640 0x7f172c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f173158f640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0c9970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f172800e7c0 tx=0x7f172800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f1721ffb640 1 -- 172.21.15.66:0/352677036 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172800ccc0 con 0x7f172c0cdc00 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f1721ffb640 1 -- 172.21.15.66:0/352677036 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1728010430 con 0x7f172c0cdc00 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f1721ffb640 1 -- 172.21.15.66:0/352677036 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728010860 con 0x7f172c0cdc00 2024-09-10T22:35:08.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.308+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c1f59b0 con 0x7f172c0cdc00 2024-09-10T22:35:08.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.309+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c0c8e90 con 0x7f172c0cdc00 2024-09-10T22:35:08.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.309+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172c0cecc0 con 0x7f172c0cdc00 2024-09-10T22:35:08.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.310+0000 7f1721ffb640 1 -- 172.21.15.66:0/352677036 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1728010a00 con 0x7f172c0cdc00 2024-09-10T22:35:08.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.311+0000 7f1721ffb640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1710077f70 0x7f171007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:08.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.311+0000 7f1721ffb640 1 -- 172.21.15.66:0/352677036 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f172809a790 con 0x7f172c0cdc00 2024-09-10T22:35:08.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.311+0000 7f1730d8e640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1710077f70 0x7f171007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:08.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.311+0000 7f1730d8e640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1710077f70 0x7f171007a430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f171c033000 tx=0x7f171c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:08.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.315+0000 7f1721ffb640 1 -- 172.21.15.66:0/352677036 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1728062bc0 con 0x7f172c0cdc00 2024-09-10T22:35:08.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.499+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f172c0cbd00 con 0x7f1710077f70 2024-09-10T22:35:08.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.504+0000 7f1721ffb640 1 -- 172.21.15.66:0/352677036 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243982 (secure 0 0 0) 0x7f172c0cbd00 con 0x7f1710077f70 2024-09-10T22:35:08.507 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:08.508 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:08.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1710077f70 msgr2=0x7f171007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1710077f70 0x7f171007a430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f171c033000 tx=0x7f171c004730 comp rx=0 tx=0).stop 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 msgr2=0x7f172c0c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0c9970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f172800e7c0 tx=0x7f172800ec80 comp rx=0 tx=0).stop 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 shutdown_connections 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1710077f70 0x7f171007a430 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f172c161640 0x7f172c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f172c0ce6a0 0x7f172c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 --2- 172.21.15.66:0/352677036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f172c0cdc00 0x7f172c0c9970 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 >> 172.21.15.66:0/352677036 conn(0x7f172c0d1d20 msgr2=0x7f172c070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 shutdown_connections 2024-09-10T22:35:08.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:08.507+0000 7f17337f3640 1 -- 172.21.15.66:0/352677036 wait complete. 2024-09-10T22:35:08.518 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T22:35:06.593306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":122,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":2608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":2670592,"data_stored":1652692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.555133"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060843+0000","last_change":"2024-09-10T22:35:03.065907+0000","last_active":"2024-09-10T22:35:04.060843+0000","last_peered":"2024-09-10T22:35:04.060843+0000","last_clean":"2024-09-10T22:35:04.060843+0000","last_became_active":"2024-09-10T22:35:03.065607+0000","last_became_peered":"2024-09-10T22:35:03.065607+0000","last_unstale":"2024-09-10T22:35:04.060843+0000","last_undegraded":"2024-09-10T22:35:04.060843+0000","last_fullsized":"2024-09-10T22:35:04.060843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283932+0000","last_change":"2024-09-10T22:35:03.072149+0000","last_active":"2024-09-10T22:35:04.283932+0000","last_peered":"2024-09-10T22:35:04.283932+0000","last_clean":"2024-09-10T22:35:04.283932+0000","last_became_active":"2024-09-10T22:35:03.071819+0000","last_became_peered":"2024-09-10T22:35:03.071819+0000","last_unstale":"2024-09-10T22:35:04.283932+0000","last_undegraded":"2024-09-10T22:35:04.283932+0000","last_fullsized":"2024-09-10T22:35:04.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:26.466821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283542+0000","last_change":"2024-09-10T22:35:03.080808+0000","last_active":"2024-09-10T22:35:04.283542+0000","last_peered":"2024-09-10T22:35:04.283542+0000","last_clean":"2024-09-10T22:35:04.283542+0000","last_became_active":"2024-09-10T22:35:03.080467+0000","last_became_peered":"2024-09-10T22:35:03.080467+0000","last_unstale":"2024-09-10T22:35:04.283542+0000","last_undegraded":"2024-09-10T22:35:04.283542+0000","last_fullsized":"2024-09-10T22:35:04.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060665+0000","last_change":"2024-09-10T22:35:03.062056+0000","last_active":"2024-09-10T22:35:04.060665+0000","last_peered":"2024-09-10T22:35:04.060665+0000","last_clean":"2024-09-10T22:35:04.060665+0000","last_became_active":"2024-09-10T22:35:03.061769+0000","last_became_peered":"2024-09-10T22:35:03.061769+0000","last_unstale":"2024-09-10T22:35:04.060665+0000","last_undegraded":"2024-09-10T22:35:04.060665+0000","last_fullsized":"2024-09-10T22:35:04.060665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:02.406217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282700+0000","last_change":"2024-09-10T22:35:03.060118+0000","last_active":"2024-09-10T22:35:04.282700+0000","last_peered":"2024-09-10T22:35:04.282700+0000","last_clean":"2024-09-10T22:35:04.282700+0000","last_became_active":"2024-09-10T22:35:03.059799+0000","last_became_peered":"2024-09-10T22:35:03.059799+0000","last_unstale":"2024-09-10T22:35:04.282700+0000","last_undegraded":"2024-09-10T22:35:04.282700+0000","last_fullsized":"2024-09-10T22:35:04.282700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:50.965802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283468+0000","last_change":"2024-09-10T22:35:03.076617+0000","last_active":"2024-09-10T22:35:04.283468+0000","last_peered":"2024-09-10T22:35:04.283468+0000","last_clean":"2024-09-10T22:35:04.283468+0000","last_became_active":"2024-09-10T22:35:03.075527+0000","last_became_peered":"2024-09-10T22:35:03.075527+0000","last_unstale":"2024-09-10T22:35:04.283468+0000","last_undegraded":"2024-09-10T22:35:04.283468+0000","last_fullsized":"2024-09-10T22:35:04.283468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283390+0000","last_change":"2024-09-10T22:35:03.076662+0000","last_active":"2024-09-10T22:35:04.283390+0000","last_peered":"2024-09-10T22:35:04.283390+0000","last_clean":"2024-09-10T22:35:04.283390+0000","last_became_active":"2024-09-10T22:35:03.075696+0000","last_became_peered":"2024-09-10T22:35:03.075696+0000","last_unstale":"2024-09-10T22:35:04.283390+0000","last_undegraded":"2024-09-10T22:35:04.283390+0000","last_fullsized":"2024-09-10T22:35:04.283390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:59.070897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060768+0000","last_change":"2024-09-10T22:35:03.074633+0000","last_active":"2024-09-10T22:35:04.060768+0000","last_peered":"2024-09-10T22:35:04.060768+0000","last_clean":"2024-09-10T22:35:04.060768+0000","last_became_active":"2024-09-10T22:35:03.074332+0000","last_became_peered":"2024-09-10T22:35:03.074332+0000","last_unstale":"2024-09-10T22:35:04.060768+0000","last_undegraded":"2024-09-10T22:35:04.060768+0000","last_fullsized":"2024-09-10T22:35:04.060768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283311+0000","last_change":"2024-09-10T22:35:03.077344+0000","last_active":"2024-09-10T22:35:04.283311+0000","last_peered":"2024-09-10T22:35:04.283311+0000","last_clean":"2024-09-10T22:35:04.283311+0000","last_became_active":"2024-09-10T22:35:03.075668+0000","last_became_peered":"2024-09-10T22:35:03.075668+0000","last_unstale":"2024-09-10T22:35:04.283311+0000","last_undegraded":"2024-09-10T22:35:04.283311+0000","last_fullsized":"2024-09-10T22:35:04.283311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282910+0000","last_change":"2024-09-10T22:35:03.058708+0000","last_active":"2024-09-10T22:35:04.282910+0000","last_peered":"2024-09-10T22:35:04.282910+0000","last_clean":"2024-09-10T22:35:04.282910+0000","last_became_active":"2024-09-10T22:35:03.058399+0000","last_became_peered":"2024-09-10T22:35:03.058399+0000","last_unstale":"2024-09-10T22:35:04.282910+0000","last_undegraded":"2024-09-10T22:35:04.282910+0000","last_fullsized":"2024-09-10T22:35:04.282910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:21.971330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283873+0000","last_change":"2024-09-10T22:35:02.060438+0000","last_active":"2024-09-10T22:35:04.283873+0000","last_peered":"2024-09-10T22:35:04.283873+0000","last_clean":"2024-09-10T22:35:04.283873+0000","last_became_active":"2024-09-10T22:35:02.060124+0000","last_became_peered":"2024-09-10T22:35:02.060124+0000","last_unstale":"2024-09-10T22:35:04.283873+0000","last_undegraded":"2024-09-10T22:35:04.283873+0000","last_fullsized":"2024-09-10T22:35:04.283873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044841+0000","last_change":"2024-09-10T22:35:01.044841+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044841+0000","last_undegraded":"2024-09-10T22:35:01.044841+0000","last_fullsized":"2024-09-10T22:35:01.044841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283633+0000","last_change":"2024-09-10T22:35:02.058735+0000","last_active":"2024-09-10T22:35:04.283633+0000","last_peered":"2024-09-10T22:35:04.283633+0000","last_clean":"2024-09-10T22:35:04.283633+0000","last_became_active":"2024-09-10T22:35:02.058425+0000","last_became_peered":"2024-09-10T22:35:02.058425+0000","last_unstale":"2024-09-10T22:35:04.283633+0000","last_undegraded":"2024-09-10T22:35:04.283633+0000","last_fullsized":"2024-09-10T22:35:04.283633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284127+0000","last_change":"2024-09-10T22:35:03.070043+0000","last_active":"2024-09-10T22:35:04.284127+0000","last_peered":"2024-09-10T22:35:04.284127+0000","last_clean":"2024-09-10T22:35:04.284127+0000","last_became_active":"2024-09-10T22:35:03.069418+0000","last_became_peered":"2024-09-10T22:35:03.069418+0000","last_unstale":"2024-09-10T22:35:04.284127+0000","last_undegraded":"2024-09-10T22:35:04.284127+0000","last_fullsized":"2024-09-10T22:35:04.284127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:23.917793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.042994+0000","last_change":"2024-09-10T22:35:01.042994+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.042994+0000","last_undegraded":"2024-09-10T22:35:01.042994+0000","last_fullsized":"2024-09-10T22:35:01.042994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060647+0000","last_change":"2024-09-10T22:35:03.074156+0000","last_active":"2024-09-10T22:35:04.060647+0000","last_peered":"2024-09-10T22:35:04.060647+0000","last_clean":"2024-09-10T22:35:04.060647+0000","last_became_active":"2024-09-10T22:35:03.073836+0000","last_became_peered":"2024-09-10T22:35:03.073836+0000","last_unstale":"2024-09-10T22:35:04.060647+0000","last_undegraded":"2024-09-10T22:35:04.060647+0000","last_fullsized":"2024-09-10T22:35:04.060647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.045510+0000","last_change":"2024-09-10T22:35:01.045510+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.045510+0000","last_undegraded":"2024-09-10T22:35:01.045510+0000","last_fullsized":"2024-09-10T22:35:01.045510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043032+0000","last_change":"2024-09-10T22:35:01.043032+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043032+0000","last_undegraded":"2024-09-10T22:35:01.043032+0000","last_fullsized":"2024-09-10T22:35:01.043032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060881+0000","last_change":"2024-09-10T22:35:02.061366+0000","last_active":"2024-09-10T22:35:04.060881+0000","last_peered":"2024-09-10T22:35:04.060881+0000","last_clean":"2024-09-10T22:35:04.060881+0000","last_became_active":"2024-09-10T22:35:02.061092+0000","last_became_peered":"2024-09-10T22:35:02.061092+0000","last_unstale":"2024-09-10T22:35:04.060881+0000","last_undegraded":"2024-09-10T22:35:04.060881+0000","last_fullsized":"2024-09-10T22:35:04.060881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044803+0000","last_change":"2024-09-10T22:35:01.044803+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044803+0000","last_undegraded":"2024-09-10T22:35:01.044803+0000","last_fullsized":"2024-09-10T22:35:01.044803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043089+0000","last_change":"2024-09-10T22:35:01.043089+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043089+0000","last_undegraded":"2024-09-10T22:35:01.043089+0000","last_fullsized":"2024-09-10T22:35:01.043089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043732+0000","last_change":"2024-09-10T22:35:01.043732+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043732+0000","last_undegraded":"2024-09-10T22:35:01.043732+0000","last_fullsized":"2024-09-10T22:35:01.043732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044346+0000","last_change":"2024-09-10T22:35:01.044346+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044346+0000","last_undegraded":"2024-09-10T22:35:01.044346+0000","last_fullsized":"2024-09-10T22:35:01.044346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282601+0000","last_change":"2024-09-10T22:35:02.058914+0000","last_active":"2024-09-10T22:35:04.282601+0000","last_peered":"2024-09-10T22:35:04.282601+0000","last_clean":"2024-09-10T22:35:04.282601+0000","last_became_active":"2024-09-10T22:35:02.058054+0000","last_became_peered":"2024-09-10T22:35:02.058054+0000","last_unstale":"2024-09-10T22:35:04.282601+0000","last_undegraded":"2024-09-10T22:35:04.282601+0000","last_fullsized":"2024-09-10T22:35:04.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:07.958631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040146+0000","last_change":"2024-09-10T22:35:01.040146+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040146+0000","last_undegraded":"2024-09-10T22:35:01.040146+0000","last_fullsized":"2024-09-10T22:35:01.040146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282840+0000","last_change":"2024-09-10T22:35:02.056593+0000","last_active":"2024-09-10T22:35:04.282840+0000","last_peered":"2024-09-10T22:35:04.282840+0000","last_clean":"2024-09-10T22:35:04.282840+0000","last_became_active":"2024-09-10T22:35:02.056253+0000","last_became_peered":"2024-09-10T22:35:02.056253+0000","last_unstale":"2024-09-10T22:35:04.282840+0000","last_undegraded":"2024-09-10T22:35:04.282840+0000","last_fullsized":"2024-09-10T22:35:04.282840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060976+0000","last_change":"2024-09-10T22:35:02.059186+0000","last_active":"2024-09-10T22:35:04.060976+0000","last_peered":"2024-09-10T22:35:04.060976+0000","last_clean":"2024-09-10T22:35:04.060976+0000","last_became_active":"2024-09-10T22:35:02.058905+0000","last_became_peered":"2024-09-10T22:35:02.058905+0000","last_unstale":"2024-09-10T22:35:04.060976+0000","last_undegraded":"2024-09-10T22:35:04.060976+0000","last_fullsized":"2024-09-10T22:35:04.060976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283210+0000","last_change":"2024-09-10T22:35:02.059788+0000","last_active":"2024-09-10T22:35:04.283210+0000","last_peered":"2024-09-10T22:35:04.283210+0000","last_clean":"2024-09-10T22:35:04.283210+0000","last_became_active":"2024-09-10T22:35:02.059419+0000","last_became_peered":"2024-09-10T22:35:02.059419+0000","last_unstale":"2024-09-10T22:35:04.283210+0000","last_undegraded":"2024-09-10T22:35:04.283210+0000","last_fullsized":"2024-09-10T22:35:04.283210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T22:35:01.042201+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:01.042201+0000","last_peered":"2024-09-10T22:35:01.042201+0000","last_clean":"2024-09-10T22:35:01.042201+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:01.042201+0000","last_undegraded":"2024-09-10T22:35:01.042201+0000","last_fullsized":"2024-09-10T22:35:01.042201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043423+0000","last_change":"2024-09-10T22:35:01.043423+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043423+0000","last_undegraded":"2024-09-10T22:35:01.043423+0000","last_fullsized":"2024-09-10T22:35:01.043423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283212+0000","last_change":"2024-09-10T22:35:03.079281+0000","last_active":"2024-09-10T22:35:04.283212+0000","last_peered":"2024-09-10T22:35:04.283212+0000","last_clean":"2024-09-10T22:35:04.283212+0000","last_became_active":"2024-09-10T22:35:03.078916+0000","last_became_peered":"2024-09-10T22:35:03.078916+0000","last_unstale":"2024-09-10T22:35:04.283212+0000","last_undegraded":"2024-09-10T22:35:04.283212+0000","last_fullsized":"2024-09-10T22:35:04.283212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041127+0000","last_change":"2024-09-10T22:35:01.041127+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041127+0000","last_undegraded":"2024-09-10T22:35:01.041127+0000","last_fullsized":"2024-09-10T22:35:01.041127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283032+0000","last_change":"2024-09-10T22:35:02.057593+0000","last_active":"2024-09-10T22:35:04.283032+0000","last_peered":"2024-09-10T22:35:04.283032+0000","last_clean":"2024-09-10T22:35:04.283032+0000","last_became_active":"2024-09-10T22:35:02.057203+0000","last_became_peered":"2024-09-10T22:35:02.057203+0000","last_unstale":"2024-09-10T22:35:04.283032+0000","last_undegraded":"2024-09-10T22:35:04.283032+0000","last_fullsized":"2024-09-10T22:35:04.283032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:49.760894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060660+0000","last_change":"2024-09-10T22:35:03.057957+0000","last_active":"2024-09-10T22:35:04.060660+0000","last_peered":"2024-09-10T22:35:04.060660+0000","last_clean":"2024-09-10T22:35:04.060660+0000","last_became_active":"2024-09-10T22:35:03.057658+0000","last_became_peered":"2024-09-10T22:35:03.057658+0000","last_unstale":"2024-09-10T22:35:04.060660+0000","last_undegraded":"2024-09-10T22:35:04.060660+0000","last_fullsized":"2024-09-10T22:35:04.060660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043022+0000","last_change":"2024-09-10T22:35:01.043022+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043022+0000","last_undegraded":"2024-09-10T22:35:01.043022+0000","last_fullsized":"2024-09-10T22:35:01.043022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060864+0000","last_change":"2024-09-10T22:35:02.063524+0000","last_active":"2024-09-10T22:35:04.060864+0000","last_peered":"2024-09-10T22:35:04.060864+0000","last_clean":"2024-09-10T22:35:04.060864+0000","last_became_active":"2024-09-10T22:35:02.063232+0000","last_became_peered":"2024-09-10T22:35:02.063232+0000","last_unstale":"2024-09-10T22:35:04.060864+0000","last_undegraded":"2024-09-10T22:35:04.060864+0000","last_fullsized":"2024-09-10T22:35:04.060864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040400+0000","last_change":"2024-09-10T22:35:01.040400+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040400+0000","last_undegraded":"2024-09-10T22:35:01.040400+0000","last_fullsized":"2024-09-10T22:35:01.040400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060999+0000","last_change":"2024-09-10T22:35:03.058800+0000","last_active":"2024-09-10T22:35:04.060999+0000","last_peered":"2024-09-10T22:35:04.060999+0000","last_clean":"2024-09-10T22:35:04.060999+0000","last_became_active":"2024-09-10T22:35:03.058503+0000","last_became_peered":"2024-09-10T22:35:03.058503+0000","last_unstale":"2024-09-10T22:35:04.060999+0000","last_undegraded":"2024-09-10T22:35:04.060999+0000","last_fullsized":"2024-09-10T22:35:04.060999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.058112+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.057838+0000","last_became_peered":"2024-09-10T22:35:02.057838+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:20.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283672+0000","last_change":"2024-09-10T22:35:03.072054+0000","last_active":"2024-09-10T22:35:04.283672+0000","last_peered":"2024-09-10T22:35:04.283672+0000","last_clean":"2024-09-10T22:35:04.283672+0000","last_became_active":"2024-09-10T22:35:03.071739+0000","last_became_peered":"2024-09-10T22:35:03.071739+0000","last_unstale":"2024-09-10T22:35:04.283672+0000","last_undegraded":"2024-09-10T22:35:04.283672+0000","last_fullsized":"2024-09-10T22:35:04.283672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.044311+0000","last_change":"2024-09-10T22:35:01.044311+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.044311+0000","last_undegraded":"2024-09-10T22:35:01.044311+0000","last_fullsized":"2024-09-10T22:35:01.044311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041610+0000","last_change":"2024-09-10T22:35:01.041610+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041610+0000","last_undegraded":"2024-09-10T22:35:01.041610+0000","last_fullsized":"2024-09-10T22:35:01.041610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060764+0000","last_change":"2024-09-10T22:35:02.061273+0000","last_active":"2024-09-10T22:35:04.060764+0000","last_peered":"2024-09-10T22:35:04.060764+0000","last_clean":"2024-09-10T22:35:04.060764+0000","last_became_active":"2024-09-10T22:35:02.060953+0000","last_became_peered":"2024-09-10T22:35:02.060953+0000","last_unstale":"2024-09-10T22:35:04.060764+0000","last_undegraded":"2024-09-10T22:35:04.060764+0000","last_fullsized":"2024-09-10T22:35:04.060764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:32.172943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284309+0000","last_change":"2024-09-10T22:35:03.078424+0000","last_active":"2024-09-10T22:35:04.284309+0000","last_peered":"2024-09-10T22:35:04.284309+0000","last_clean":"2024-09-10T22:35:04.284309+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.284309+0000","last_undegraded":"2024-09-10T22:35:04.284309+0000","last_fullsized":"2024-09-10T22:35:04.284309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:48.985804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.061749+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.061248+0000","last_became_peered":"2024-09-10T22:35:02.061248+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:49.960770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283758+0000","last_change":"2024-09-10T22:35:03.076736+0000","last_active":"2024-09-10T22:35:04.283758+0000","last_peered":"2024-09-10T22:35:04.283758+0000","last_clean":"2024-09-10T22:35:04.283758+0000","last_became_active":"2024-09-10T22:35:03.075379+0000","last_became_peered":"2024-09-10T22:35:03.075379+0000","last_unstale":"2024-09-10T22:35:04.283758+0000","last_undegraded":"2024-09-10T22:35:04.283758+0000","last_fullsized":"2024-09-10T22:35:04.283758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.399411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040216+0000","last_change":"2024-09-10T22:35:01.040216+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040216+0000","last_undegraded":"2024-09-10T22:35:01.040216+0000","last_fullsized":"2024-09-10T22:35:01.040216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.040170+0000","last_change":"2024-09-10T22:35:01.040170+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.040170+0000","last_undegraded":"2024-09-10T22:35:01.040170+0000","last_fullsized":"2024-09-10T22:35:01.040170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.041317+0000","last_change":"2024-09-10T22:35:01.041317+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.041317+0000","last_undegraded":"2024-09-10T22:35:01.041317+0000","last_fullsized":"2024-09-10T22:35:01.041317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T22:35:02.037934+0000","last_change":"2024-09-10T22:35:02.037934+0000","last_active":"2024-09-10T22:35:02.037934+0000","last_peered":"2024-09-10T22:35:02.037934+0000","last_clean":"2024-09-10T22:35:02.037934+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:02.037934+0000","last_undegraded":"2024-09-10T22:35:02.037934+0000","last_fullsized":"2024-09-10T22:35:02.037934+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283509+0000","last_change":"2024-09-10T22:35:02.062827+0000","last_active":"2024-09-10T22:35:04.283509+0000","last_peered":"2024-09-10T22:35:04.283509+0000","last_clean":"2024-09-10T22:35:04.283509+0000","last_became_active":"2024-09-10T22:35:02.062342+0000","last_became_peered":"2024-09-10T22:35:02.062342+0000","last_unstale":"2024-09-10T22:35:04.283509+0000","last_undegraded":"2024-09-10T22:35:04.283509+0000","last_fullsized":"2024-09-10T22:35:04.283509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283072+0000","last_change":"2024-09-10T22:35:03.080815+0000","last_active":"2024-09-10T22:35:04.283072+0000","last_peered":"2024-09-10T22:35:04.283072+0000","last_clean":"2024-09-10T22:35:04.283072+0000","last_became_active":"2024-09-10T22:35:03.080484+0000","last_became_peered":"2024-09-10T22:35:03.080484+0000","last_unstale":"2024-09-10T22:35:04.283072+0000","last_undegraded":"2024-09-10T22:35:04.283072+0000","last_fullsized":"2024-09-10T22:35:04.283072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:58.700652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T22:35:01.043161+0000","last_change":"2024-09-10T22:35:01.043161+0000","last_active":"2024-09-10T22:35:01.032902+0000","last_peered":"2024-09-10T22:35:01.032902+0000","last_clean":"2024-09-10T22:35:01.032902+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T22:35:01.043161+0000","last_undegraded":"2024-09-10T22:35:01.043161+0000","last_fullsized":"2024-09-10T22:35:01.043161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":38,"acting":38,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":172032,"data_stored":52971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":753664,"data_stored":637522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:11.523 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 -- 172.21.15.66:0/3007101705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55415be20 msgr2=0x7fa55406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 --2- 172.21.15.66:0/3007101705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55415be20 0x7fa55406cb20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5540b6a80 tx=0x7fa54402a5f0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 -- 172.21.15.66:0/3007101705 shutdown_connections 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 --2- 172.21.15.66:0/3007101705 >> v2:172.21.15.94:3300/0 conn(0x7fa554071b80 0x7fa55415f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 --2- 172.21.15.66:0/3007101705 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa55406d060 0x7fa554071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 --2- 172.21.15.66:0/3007101705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55415be20 0x7fa55406cb20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 -- 172.21.15.66:0/3007101705 >> 172.21.15.66:0/3007101705 conn(0x7fa5540d0c90 msgr2=0x7fa5540d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 -- 172.21.15.66:0/3007101705 shutdown_connections 2024-09-10T22:35:11.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.640+0000 7fa558a23640 1 -- 172.21.15.66:0/3007101705 wait complete. 2024-09-10T22:35:11.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.642+0000 7fa558a23640 1 Processor -- start 2024-09-10T22:35:11.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.642+0000 7fa558a23640 1 -- start start 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa558a23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55406d060 0x7fa5540d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa558a23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa554071b80 0x7fa554151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa558a23640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5540cfa00 0x7fa5540c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa558a23640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa55415fd40 con 0x7fa55406d060 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa558a23640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa55415fec0 con 0x7fa554071b80 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa558a23640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa554160040 con 0x7fa5540cfa00 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55406d060 0x7fa5540d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55406d060 0x7fa5540d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55766/0 (socket says 172.21.15.66:55766) 2024-09-10T22:35:11.645 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 -- 172.21.15.66:0/3019971822 learned_addr learned my addr 172.21.15.66:0/3019971822 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:11.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa552d9d640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5540cfa00 0x7fa5540c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:11.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa551d9b640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa554071b80 0x7fa554151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:11.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 -- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5540cfa00 msgr2=0x7fa5540c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:11.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5540cfa00 0x7fa5540c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 -- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa554071b80 msgr2=0x7fa554151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:11.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa554071b80 0x7fa554151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 -- 172.21.15.66:0/3019971822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5540ad8f0 con 0x7fa55406d060 2024-09-10T22:35:11.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.643+0000 7fa55259c640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55406d060 0x7fa5540d04d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa544034040 tx=0x7fa5440038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:11.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.644+0000 7fa542ffd640 1 -- 172.21.15.66:0/3019971822 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54403e030 con 0x7fa55406d060 2024-09-10T22:35:11.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.644+0000 7fa542ffd640 1 -- 172.21.15.66:0/3019971822 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa544003ca0 con 0x7fa55406d060 2024-09-10T22:35:11.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.644+0000 7fa542ffd640 1 -- 172.21.15.66:0/3019971822 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544035040 con 0x7fa55406d060 2024-09-10T22:35:11.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.644+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5541f8610 con 0x7fa55406d060 2024-09-10T22:35:11.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.644+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5541f8bd0 con 0x7fa55406d060 2024-09-10T22:35:11.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.645+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5541f93b0 con 0x7fa55406d060 2024-09-10T22:35:11.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.645+0000 7fa542ffd640 1 -- 172.21.15.66:0/3019971822 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa544004a00 con 0x7fa55406d060 2024-09-10T22:35:11.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.646+0000 7fa542ffd640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa534077f70 0x7fa53407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:11.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.646+0000 7fa542ffd640 1 -- 172.21.15.66:0/3019971822 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa5440ba4d0 con 0x7fa55406d060 2024-09-10T22:35:11.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.646+0000 7fa551d9b640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa534077f70 0x7fa53407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:11.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.647+0000 7fa551d9b640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa534077f70 0x7fa53407a430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5541588b0 tx=0x7fa548005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:11.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.650+0000 7fa542ffd640 1 -- 172.21.15.66:0/3019971822 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa544082a10 con 0x7fa55406d060 2024-09-10T22:35:11.832 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.831+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5540d01a0 con 0x7fa534077f70 2024-09-10T22:35:11.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.836+0000 7fa542ffd640 1 -- 172.21.15.66:0/3019971822 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7fa548011bb0 con 0x7fa534077f70 2024-09-10T22:35:11.840 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:11.841 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa534077f70 msgr2=0x7fa53407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa534077f70 0x7fa53407a430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5541588b0 tx=0x7fa548005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55406d060 msgr2=0x7fa5540d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55406d060 0x7fa5540d04d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa544034040 tx=0x7fa5440038a0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 shutdown_connections 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa534077f70 0x7fa53407a430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa5540cfa00 0x7fa5540c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa554071b80 0x7fa554151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 --2- 172.21.15.66:0/3019971822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa55406d060 0x7fa5540d04d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 >> 172.21.15.66:0/3019971822 conn(0x7fa5540d0c90 msgr2=0x7fa5540b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:11.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 shutdown_connections 2024-09-10T22:35:11.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.840+0000 7fa558a23640 1 -- 172.21.15.66:0/3019971822 wait complete. 2024-09-10T22:35:11.852 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-10T22:35:10.594633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235760,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738960,"statfs":{"total":773094113280,"available":772852695040,"internally_reserved":0,"allocated":2924544,"data_stored":1764124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002449"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283978+0000","last_change":"2024-09-10T22:35:03.064865+0000","last_active":"2024-09-10T22:35:04.283978+0000","last_peered":"2024-09-10T22:35:04.283978+0000","last_clean":"2024-09-10T22:35:04.283978+0000","last_became_active":"2024-09-10T22:35:03.064552+0000","last_became_peered":"2024-09-10T22:35:03.064552+0000","last_unstale":"2024-09-10T22:35:04.283978+0000","last_undegraded":"2024-09-10T22:35:04.283978+0000","last_fullsized":"2024-09-10T22:35:04.283978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:42.990367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060843+0000","last_change":"2024-09-10T22:35:03.065907+0000","last_active":"2024-09-10T22:35:04.060843+0000","last_peered":"2024-09-10T22:35:04.060843+0000","last_clean":"2024-09-10T22:35:04.060843+0000","last_became_active":"2024-09-10T22:35:03.065607+0000","last_became_peered":"2024-09-10T22:35:03.065607+0000","last_unstale":"2024-09-10T22:35:04.060843+0000","last_undegraded":"2024-09-10T22:35:04.060843+0000","last_fullsized":"2024-09-10T22:35:04.060843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283605+0000","last_change":"2024-09-10T22:35:03.063096+0000","last_active":"2024-09-10T22:35:04.283605+0000","last_peered":"2024-09-10T22:35:04.283605+0000","last_clean":"2024-09-10T22:35:04.283605+0000","last_became_active":"2024-09-10T22:35:03.057986+0000","last_became_peered":"2024-09-10T22:35:03.057986+0000","last_unstale":"2024-09-10T22:35:04.283605+0000","last_undegraded":"2024-09-10T22:35:04.283605+0000","last_fullsized":"2024-09-10T22:35:04.283605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:41.190200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062415+0000","last_change":"2024-09-10T22:35:03.076412+0000","last_active":"2024-09-10T22:35:04.062415+0000","last_peered":"2024-09-10T22:35:04.062415+0000","last_clean":"2024-09-10T22:35:04.062415+0000","last_became_active":"2024-09-10T22:35:03.076037+0000","last_became_peered":"2024-09-10T22:35:03.076037+0000","last_unstale":"2024-09-10T22:35:04.062415+0000","last_undegraded":"2024-09-10T22:35:04.062415+0000","last_fullsized":"2024-09-10T22:35:04.062415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:35.334336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284073+0000","last_change":"2024-09-10T22:35:03.061394+0000","last_active":"2024-09-10T22:35:04.284073+0000","last_peered":"2024-09-10T22:35:04.284073+0000","last_clean":"2024-09-10T22:35:04.284073+0000","last_became_active":"2024-09-10T22:35:03.061111+0000","last_became_peered":"2024-09-10T22:35:03.061111+0000","last_unstale":"2024-09-10T22:35:04.284073+0000","last_undegraded":"2024-09-10T22:35:04.284073+0000","last_fullsized":"2024-09-10T22:35:04.284073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:48.372055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062681+0000","last_change":"2024-09-10T22:35:03.062367+0000","last_active":"2024-09-10T22:35:04.062681+0000","last_peered":"2024-09-10T22:35:04.062681+0000","last_clean":"2024-09-10T22:35:04.062681+0000","last_became_active":"2024-09-10T22:35:03.061999+0000","last_became_peered":"2024-09-10T22:35:03.061999+0000","last_unstale":"2024-09-10T22:35:04.062681+0000","last_undegraded":"2024-09-10T22:35:04.062681+0000","last_fullsized":"2024-09-10T22:35:04.062681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:42.990367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282913+0000","last_change":"2024-09-10T22:35:03.065678+0000","last_active":"2024-09-10T22:35:04.282913+0000","last_peered":"2024-09-10T22:35:04.282913+0000","last_clean":"2024-09-10T22:35:04.282913+0000","last_became_active":"2024-09-10T22:35:03.065414+0000","last_became_peered":"2024-09-10T22:35:03.065414+0000","last_unstale":"2024-09-10T22:35:04.282913+0000","last_undegraded":"2024-09-10T22:35:04.282913+0000","last_fullsized":"2024-09-10T22:35:04.282913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:56.275057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283932+0000","last_change":"2024-09-10T22:35:03.072149+0000","last_active":"2024-09-10T22:35:04.283932+0000","last_peered":"2024-09-10T22:35:04.283932+0000","last_clean":"2024-09-10T22:35:04.283932+0000","last_became_active":"2024-09-10T22:35:03.071819+0000","last_became_peered":"2024-09-10T22:35:03.071819+0000","last_unstale":"2024-09-10T22:35:04.283932+0000","last_undegraded":"2024-09-10T22:35:04.283932+0000","last_fullsized":"2024-09-10T22:35:04.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:26.466821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284074+0000","last_change":"2024-09-10T22:35:03.068359+0000","last_active":"2024-09-10T22:35:04.284074+0000","last_peered":"2024-09-10T22:35:04.284074+0000","last_clean":"2024-09-10T22:35:04.284074+0000","last_became_active":"2024-09-10T22:35:03.067889+0000","last_became_peered":"2024-09-10T22:35:03.067889+0000","last_unstale":"2024-09-10T22:35:04.284074+0000","last_undegraded":"2024-09-10T22:35:04.284074+0000","last_fullsized":"2024-09-10T22:35:04.284074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:06.735630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283225+0000","last_change":"2024-09-10T22:35:03.061509+0000","last_active":"2024-09-10T22:35:04.283225+0000","last_peered":"2024-09-10T22:35:04.283225+0000","last_clean":"2024-09-10T22:35:04.283225+0000","last_became_active":"2024-09-10T22:35:03.061170+0000","last_became_peered":"2024-09-10T22:35:03.061170+0000","last_unstale":"2024-09-10T22:35:04.283225+0000","last_undegraded":"2024-09-10T22:35:04.283225+0000","last_fullsized":"2024-09-10T22:35:04.283225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:26.993716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283408+0000","last_change":"2024-09-10T22:35:03.061387+0000","last_active":"2024-09-10T22:35:04.283408+0000","last_peered":"2024-09-10T22:35:04.283408+0000","last_clean":"2024-09-10T22:35:04.283408+0000","last_became_active":"2024-09-10T22:35:03.061088+0000","last_became_peered":"2024-09-10T22:35:03.061088+0000","last_unstale":"2024-09-10T22:35:04.283408+0000","last_undegraded":"2024-09-10T22:35:04.283408+0000","last_fullsized":"2024-09-10T22:35:04.283408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:31.398073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063686+0000","last_change":"2024-09-10T22:35:03.058702+0000","last_active":"2024-09-10T22:35:04.063686+0000","last_peered":"2024-09-10T22:35:04.063686+0000","last_clean":"2024-09-10T22:35:04.063686+0000","last_became_active":"2024-09-10T22:35:03.058419+0000","last_became_peered":"2024-09-10T22:35:03.058419+0000","last_unstale":"2024-09-10T22:35:04.063686+0000","last_undegraded":"2024-09-10T22:35:04.063686+0000","last_fullsized":"2024-09-10T22:35:04.063686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:59.284853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283542+0000","last_change":"2024-09-10T22:35:03.080808+0000","last_active":"2024-09-10T22:35:04.283542+0000","last_peered":"2024-09-10T22:35:04.283542+0000","last_clean":"2024-09-10T22:35:04.283542+0000","last_became_active":"2024-09-10T22:35:03.080467+0000","last_became_peered":"2024-09-10T22:35:03.080467+0000","last_unstale":"2024-09-10T22:35:04.283542+0000","last_undegraded":"2024-09-10T22:35:04.283542+0000","last_fullsized":"2024-09-10T22:35:04.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282815+0000","last_change":"2024-09-10T22:35:03.057789+0000","last_active":"2024-09-10T22:35:04.282815+0000","last_peered":"2024-09-10T22:35:04.282815+0000","last_clean":"2024-09-10T22:35:04.282815+0000","last_became_active":"2024-09-10T22:35:03.057450+0000","last_became_peered":"2024-09-10T22:35:03.057450+0000","last_unstale":"2024-09-10T22:35:04.282815+0000","last_undegraded":"2024-09-10T22:35:04.282815+0000","last_fullsized":"2024-09-10T22:35:04.282815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:30.343040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060665+0000","last_change":"2024-09-10T22:35:03.062056+0000","last_active":"2024-09-10T22:35:04.060665+0000","last_peered":"2024-09-10T22:35:04.060665+0000","last_clean":"2024-09-10T22:35:04.060665+0000","last_became_active":"2024-09-10T22:35:03.061769+0000","last_became_peered":"2024-09-10T22:35:03.061769+0000","last_unstale":"2024-09-10T22:35:04.060665+0000","last_undegraded":"2024-09-10T22:35:04.060665+0000","last_fullsized":"2024-09-10T22:35:04.060665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:02.406217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.061953+0000","last_change":"2024-09-10T22:35:03.067251+0000","last_active":"2024-09-10T22:35:04.061953+0000","last_peered":"2024-09-10T22:35:04.061953+0000","last_clean":"2024-09-10T22:35:04.061953+0000","last_became_active":"2024-09-10T22:35:03.066980+0000","last_became_peered":"2024-09-10T22:35:03.066980+0000","last_unstale":"2024-09-10T22:35:04.061953+0000","last_undegraded":"2024-09-10T22:35:04.061953+0000","last_fullsized":"2024-09-10T22:35:04.061953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:45.349941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062881+0000","last_change":"2024-09-10T22:35:03.070627+0000","last_active":"2024-09-10T22:35:04.062881+0000","last_peered":"2024-09-10T22:35:04.062881+0000","last_clean":"2024-09-10T22:35:04.062881+0000","last_became_active":"2024-09-10T22:35:03.070365+0000","last_became_peered":"2024-09-10T22:35:03.070365+0000","last_unstale":"2024-09-10T22:35:04.062881+0000","last_undegraded":"2024-09-10T22:35:04.062881+0000","last_fullsized":"2024-09-10T22:35:04.062881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:56.758850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282700+0000","last_change":"2024-09-10T22:35:03.060118+0000","last_active":"2024-09-10T22:35:04.282700+0000","last_peered":"2024-09-10T22:35:04.282700+0000","last_clean":"2024-09-10T22:35:04.282700+0000","last_became_active":"2024-09-10T22:35:03.059799+0000","last_became_peered":"2024-09-10T22:35:03.059799+0000","last_unstale":"2024-09-10T22:35:04.282700+0000","last_undegraded":"2024-09-10T22:35:04.282700+0000","last_fullsized":"2024-09-10T22:35:04.282700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:50.965802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284045+0000","last_change":"2024-09-10T22:35:03.062861+0000","last_active":"2024-09-10T22:35:04.284045+0000","last_peered":"2024-09-10T22:35:04.284045+0000","last_clean":"2024-09-10T22:35:04.284045+0000","last_became_active":"2024-09-10T22:35:03.062586+0000","last_became_peered":"2024-09-10T22:35:03.062586+0000","last_unstale":"2024-09-10T22:35:04.284045+0000","last_undegraded":"2024-09-10T22:35:04.284045+0000","last_fullsized":"2024-09-10T22:35:04.284045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:22.142634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283055+0000","last_change":"2024-09-10T22:35:03.070244+0000","last_active":"2024-09-10T22:35:04.283055+0000","last_peered":"2024-09-10T22:35:04.283055+0000","last_clean":"2024-09-10T22:35:04.283055+0000","last_became_active":"2024-09-10T22:35:03.067927+0000","last_became_peered":"2024-09-10T22:35:03.067927+0000","last_unstale":"2024-09-10T22:35:04.283055+0000","last_undegraded":"2024-09-10T22:35:04.283055+0000","last_fullsized":"2024-09-10T22:35:04.283055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:51.838173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283820+0000","last_change":"2024-09-10T22:35:03.065175+0000","last_active":"2024-09-10T22:35:04.283820+0000","last_peered":"2024-09-10T22:35:04.283820+0000","last_clean":"2024-09-10T22:35:04.283820+0000","last_became_active":"2024-09-10T22:35:03.064880+0000","last_became_peered":"2024-09-10T22:35:03.064880+0000","last_unstale":"2024-09-10T22:35:04.283820+0000","last_undegraded":"2024-09-10T22:35:04.283820+0000","last_fullsized":"2024-09-10T22:35:04.283820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:34.515907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062375+0000","last_change":"2024-09-10T22:35:03.069577+0000","last_active":"2024-09-10T22:35:04.062375+0000","last_peered":"2024-09-10T22:35:04.062375+0000","last_clean":"2024-09-10T22:35:04.062375+0000","last_became_active":"2024-09-10T22:35:03.069224+0000","last_became_peered":"2024-09-10T22:35:03.069224+0000","last_unstale":"2024-09-10T22:35:04.062375+0000","last_undegraded":"2024-09-10T22:35:04.062375+0000","last_fullsized":"2024-09-10T22:35:04.062375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:16.644778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283468+0000","last_change":"2024-09-10T22:35:03.076617+0000","last_active":"2024-09-10T22:35:04.283468+0000","last_peered":"2024-09-10T22:35:04.283468+0000","last_clean":"2024-09-10T22:35:04.283468+0000","last_became_active":"2024-09-10T22:35:03.075527+0000","last_became_peered":"2024-09-10T22:35:03.075527+0000","last_unstale":"2024-09-10T22:35:04.283468+0000","last_undegraded":"2024-09-10T22:35:04.283468+0000","last_fullsized":"2024-09-10T22:35:04.283468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062216+0000","last_change":"2024-09-10T22:35:03.072643+0000","last_active":"2024-09-10T22:35:04.062216+0000","last_peered":"2024-09-10T22:35:04.062216+0000","last_clean":"2024-09-10T22:35:04.062216+0000","last_became_active":"2024-09-10T22:35:03.072354+0000","last_became_peered":"2024-09-10T22:35:03.072354+0000","last_unstale":"2024-09-10T22:35:04.062216+0000","last_undegraded":"2024-09-10T22:35:04.062216+0000","last_fullsized":"2024-09-10T22:35:04.062216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:25.832743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283390+0000","last_change":"2024-09-10T22:35:03.076662+0000","last_active":"2024-09-10T22:35:04.283390+0000","last_peered":"2024-09-10T22:35:04.283390+0000","last_clean":"2024-09-10T22:35:04.283390+0000","last_became_active":"2024-09-10T22:35:03.075696+0000","last_became_peered":"2024-09-10T22:35:03.075696+0000","last_unstale":"2024-09-10T22:35:04.283390+0000","last_undegraded":"2024-09-10T22:35:04.283390+0000","last_fullsized":"2024-09-10T22:35:04.283390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:59.070897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283416+0000","last_change":"2024-09-10T22:35:03.070886+0000","last_active":"2024-09-10T22:35:04.283416+0000","last_peered":"2024-09-10T22:35:04.283416+0000","last_clean":"2024-09-10T22:35:04.283416+0000","last_became_active":"2024-09-10T22:35:03.070569+0000","last_became_peered":"2024-09-10T22:35:03.070569+0000","last_unstale":"2024-09-10T22:35:04.283416+0000","last_undegraded":"2024-09-10T22:35:04.283416+0000","last_fullsized":"2024-09-10T22:35:04.283416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:25.138442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060768+0000","last_change":"2024-09-10T22:35:03.074633+0000","last_active":"2024-09-10T22:35:04.060768+0000","last_peered":"2024-09-10T22:35:04.060768+0000","last_clean":"2024-09-10T22:35:04.060768+0000","last_became_active":"2024-09-10T22:35:03.074332+0000","last_became_peered":"2024-09-10T22:35:03.074332+0000","last_unstale":"2024-09-10T22:35:04.060768+0000","last_undegraded":"2024-09-10T22:35:04.060768+0000","last_fullsized":"2024-09-10T22:35:04.060768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283311+0000","last_change":"2024-09-10T22:35:03.077344+0000","last_active":"2024-09-10T22:35:04.283311+0000","last_peered":"2024-09-10T22:35:04.283311+0000","last_clean":"2024-09-10T22:35:04.283311+0000","last_became_active":"2024-09-10T22:35:03.075668+0000","last_became_peered":"2024-09-10T22:35:03.075668+0000","last_unstale":"2024-09-10T22:35:04.283311+0000","last_undegraded":"2024-09-10T22:35:04.283311+0000","last_fullsized":"2024-09-10T22:35:04.283311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283878+0000","last_change":"2024-09-10T22:35:03.078565+0000","last_active":"2024-09-10T22:35:04.283878+0000","last_peered":"2024-09-10T22:35:04.283878+0000","last_clean":"2024-09-10T22:35:04.283878+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.283878+0000","last_undegraded":"2024-09-10T22:35:04.283878+0000","last_fullsized":"2024-09-10T22:35:04.283878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:57.500600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062623+0000","last_change":"2024-09-10T22:35:03.075919+0000","last_active":"2024-09-10T22:35:04.062623+0000","last_peered":"2024-09-10T22:35:04.062623+0000","last_clean":"2024-09-10T22:35:04.062623+0000","last_became_active":"2024-09-10T22:35:03.075584+0000","last_became_peered":"2024-09-10T22:35:03.075584+0000","last_unstale":"2024-09-10T22:35:04.062623+0000","last_undegraded":"2024-09-10T22:35:04.062623+0000","last_fullsized":"2024-09-10T22:35:04.062623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:57.797898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282910+0000","last_change":"2024-09-10T22:35:03.058708+0000","last_active":"2024-09-10T22:35:04.282910+0000","last_peered":"2024-09-10T22:35:04.282910+0000","last_clean":"2024-09-10T22:35:04.282910+0000","last_became_active":"2024-09-10T22:35:03.058399+0000","last_became_peered":"2024-09-10T22:35:03.058399+0000","last_unstale":"2024-09-10T22:35:04.282910+0000","last_undegraded":"2024-09-10T22:35:04.282910+0000","last_fullsized":"2024-09-10T22:35:04.282910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:21.971330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282922+0000","last_change":"2024-09-10T22:35:03.066152+0000","last_active":"2024-09-10T22:35:04.282922+0000","last_peered":"2024-09-10T22:35:04.282922+0000","last_clean":"2024-09-10T22:35:04.282922+0000","last_became_active":"2024-09-10T22:35:03.065897+0000","last_became_peered":"2024-09-10T22:35:03.065897+0000","last_unstale":"2024-09-10T22:35:04.282922+0000","last_undegraded":"2024-09-10T22:35:04.282922+0000","last_fullsized":"2024-09-10T22:35:04.282922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:01.896261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063193+0000","last_change":"2024-09-10T22:35:03.062529+0000","last_active":"2024-09-10T22:35:04.063193+0000","last_peered":"2024-09-10T22:35:04.063193+0000","last_clean":"2024-09-10T22:35:04.063193+0000","last_became_active":"2024-09-10T22:35:03.059720+0000","last_became_peered":"2024-09-10T22:35:03.059720+0000","last_unstale":"2024-09-10T22:35:04.063193+0000","last_undegraded":"2024-09-10T22:35:04.063193+0000","last_fullsized":"2024-09-10T22:35:04.063193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:45.451161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283873+0000","last_change":"2024-09-10T22:35:02.060438+0000","last_active":"2024-09-10T22:35:04.283873+0000","last_peered":"2024-09-10T22:35:04.283873+0000","last_clean":"2024-09-10T22:35:04.283873+0000","last_became_active":"2024-09-10T22:35:02.060124+0000","last_became_peered":"2024-09-10T22:35:02.060124+0000","last_unstale":"2024-09-10T22:35:04.283873+0000","last_undegraded":"2024-09-10T22:35:04.283873+0000","last_fullsized":"2024-09-10T22:35:04.283873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283981+0000","last_change":"2024-09-10T22:35:03.068471+0000","last_active":"2024-09-10T22:35:04.283981+0000","last_peered":"2024-09-10T22:35:04.283981+0000","last_clean":"2024-09-10T22:35:04.283981+0000","last_became_active":"2024-09-10T22:35:03.068079+0000","last_became_peered":"2024-09-10T22:35:03.068079+0000","last_unstale":"2024-09-10T22:35:04.283981+0000","last_undegraded":"2024-09-10T22:35:04.283981+0000","last_fullsized":"2024-09-10T22:35:04.283981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:53.676311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283156+0000","last_change":"2024-09-10T22:35:02.060297+0000","last_active":"2024-09-10T22:35:04.283156+0000","last_peered":"2024-09-10T22:35:04.283156+0000","last_clean":"2024-09-10T22:35:04.283156+0000","last_became_active":"2024-09-10T22:35:02.059966+0000","last_became_peered":"2024-09-10T22:35:02.059966+0000","last_unstale":"2024-09-10T22:35:04.283156+0000","last_undegraded":"2024-09-10T22:35:04.283156+0000","last_fullsized":"2024-09-10T22:35:04.283156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.073637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283633+0000","last_change":"2024-09-10T22:35:02.058735+0000","last_active":"2024-09-10T22:35:04.283633+0000","last_peered":"2024-09-10T22:35:04.283633+0000","last_clean":"2024-09-10T22:35:04.283633+0000","last_became_active":"2024-09-10T22:35:02.058425+0000","last_became_peered":"2024-09-10T22:35:02.058425+0000","last_unstale":"2024-09-10T22:35:04.283633+0000","last_undegraded":"2024-09-10T22:35:04.283633+0000","last_fullsized":"2024-09-10T22:35:04.283633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284127+0000","last_change":"2024-09-10T22:35:03.070043+0000","last_active":"2024-09-10T22:35:04.284127+0000","last_peered":"2024-09-10T22:35:04.284127+0000","last_clean":"2024-09-10T22:35:04.284127+0000","last_became_active":"2024-09-10T22:35:03.069418+0000","last_became_peered":"2024-09-10T22:35:03.069418+0000","last_unstale":"2024-09-10T22:35:04.284127+0000","last_undegraded":"2024-09-10T22:35:04.284127+0000","last_fullsized":"2024-09-10T22:35:04.284127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:23.917793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062027+0000","last_change":"2024-09-10T22:35:02.056871+0000","last_active":"2024-09-10T22:35:04.062027+0000","last_peered":"2024-09-10T22:35:04.062027+0000","last_clean":"2024-09-10T22:35:04.062027+0000","last_became_active":"2024-09-10T22:35:02.056541+0000","last_became_peered":"2024-09-10T22:35:02.056541+0000","last_unstale":"2024-09-10T22:35:04.062027+0000","last_undegraded":"2024-09-10T22:35:04.062027+0000","last_fullsized":"2024-09-10T22:35:04.062027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:58.279821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060647+0000","last_change":"2024-09-10T22:35:03.074156+0000","last_active":"2024-09-10T22:35:04.060647+0000","last_peered":"2024-09-10T22:35:04.060647+0000","last_clean":"2024-09-10T22:35:04.060647+0000","last_became_active":"2024-09-10T22:35:03.073836+0000","last_became_peered":"2024-09-10T22:35:03.073836+0000","last_unstale":"2024-09-10T22:35:04.060647+0000","last_undegraded":"2024-09-10T22:35:04.060647+0000","last_fullsized":"2024-09-10T22:35:04.060647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283581+0000","last_change":"2024-09-10T22:35:02.063854+0000","last_active":"2024-09-10T22:35:04.283581+0000","last_peered":"2024-09-10T22:35:04.283581+0000","last_clean":"2024-09-10T22:35:04.283581+0000","last_became_active":"2024-09-10T22:35:02.063214+0000","last_became_peered":"2024-09-10T22:35:02.063214+0000","last_unstale":"2024-09-10T22:35:04.283581+0000","last_undegraded":"2024-09-10T22:35:04.283581+0000","last_fullsized":"2024-09-10T22:35:04.283581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:23.311121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062461+0000","last_change":"2024-09-10T22:35:03.069518+0000","last_active":"2024-09-10T22:35:04.062461+0000","last_peered":"2024-09-10T22:35:04.062461+0000","last_clean":"2024-09-10T22:35:04.062461+0000","last_became_active":"2024-09-10T22:35:03.069271+0000","last_became_peered":"2024-09-10T22:35:03.069271+0000","last_unstale":"2024-09-10T22:35:04.062461+0000","last_undegraded":"2024-09-10T22:35:04.062461+0000","last_fullsized":"2024-09-10T22:35:04.062461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:28.595416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062604+0000","last_change":"2024-09-10T22:35:02.060887+0000","last_active":"2024-09-10T22:35:04.062604+0000","last_peered":"2024-09-10T22:35:04.062604+0000","last_clean":"2024-09-10T22:35:04.062604+0000","last_became_active":"2024-09-10T22:35:02.060054+0000","last_became_peered":"2024-09-10T22:35:02.060054+0000","last_unstale":"2024-09-10T22:35:04.062604+0000","last_undegraded":"2024-09-10T22:35:04.062604+0000","last_fullsized":"2024-09-10T22:35:04.062604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:32.374399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283914+0000","last_change":"2024-09-10T22:35:03.063406+0000","last_active":"2024-09-10T22:35:04.283914+0000","last_peered":"2024-09-10T22:35:04.283914+0000","last_clean":"2024-09-10T22:35:04.283914+0000","last_became_active":"2024-09-10T22:35:03.062960+0000","last_became_peered":"2024-09-10T22:35:03.062960+0000","last_unstale":"2024-09-10T22:35:04.283914+0000","last_undegraded":"2024-09-10T22:35:04.283914+0000","last_fullsized":"2024-09-10T22:35:04.283914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:08.845668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060881+0000","last_change":"2024-09-10T22:35:02.061366+0000","last_active":"2024-09-10T22:35:04.060881+0000","last_peered":"2024-09-10T22:35:04.060881+0000","last_clean":"2024-09-10T22:35:04.060881+0000","last_became_active":"2024-09-10T22:35:02.061092+0000","last_became_peered":"2024-09-10T22:35:02.061092+0000","last_unstale":"2024-09-10T22:35:04.060881+0000","last_undegraded":"2024-09-10T22:35:04.060881+0000","last_fullsized":"2024-09-10T22:35:04.060881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063828+0000","last_change":"2024-09-10T22:35:03.066189+0000","last_active":"2024-09-10T22:35:04.063828+0000","last_peered":"2024-09-10T22:35:04.063828+0000","last_clean":"2024-09-10T22:35:04.063828+0000","last_became_active":"2024-09-10T22:35:03.064797+0000","last_became_peered":"2024-09-10T22:35:03.064797+0000","last_unstale":"2024-09-10T22:35:04.063828+0000","last_undegraded":"2024-09-10T22:35:04.063828+0000","last_fullsized":"2024-09-10T22:35:04.063828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:33.379430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283111+0000","last_change":"2024-09-10T22:35:02.056652+0000","last_active":"2024-09-10T22:35:04.283111+0000","last_peered":"2024-09-10T22:35:04.283111+0000","last_clean":"2024-09-10T22:35:04.283111+0000","last_became_active":"2024-09-10T22:35:02.056381+0000","last_became_peered":"2024-09-10T22:35:02.056381+0000","last_unstale":"2024-09-10T22:35:04.283111+0000","last_undegraded":"2024-09-10T22:35:04.283111+0000","last_fullsized":"2024-09-10T22:35:04.283111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:42.884854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283576+0000","last_change":"2024-09-10T22:35:03.071792+0000","last_active":"2024-09-10T22:35:04.283576+0000","last_peered":"2024-09-10T22:35:04.283576+0000","last_clean":"2024-09-10T22:35:04.283576+0000","last_became_active":"2024-09-10T22:35:03.071456+0000","last_became_peered":"2024-09-10T22:35:03.071456+0000","last_unstale":"2024-09-10T22:35:04.283576+0000","last_undegraded":"2024-09-10T22:35:04.283576+0000","last_fullsized":"2024-09-10T22:35:04.283576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:35.334336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283948+0000","last_change":"2024-09-10T22:35:02.062689+0000","last_active":"2024-09-10T22:35:04.283948+0000","last_peered":"2024-09-10T22:35:04.283948+0000","last_clean":"2024-09-10T22:35:04.283948+0000","last_became_active":"2024-09-10T22:35:02.062395+0000","last_became_peered":"2024-09-10T22:35:02.062395+0000","last_unstale":"2024-09-10T22:35:04.283948+0000","last_undegraded":"2024-09-10T22:35:04.283948+0000","last_fullsized":"2024-09-10T22:35:04.283948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:10.653575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283121+0000","last_change":"2024-09-10T22:35:03.062742+0000","last_active":"2024-09-10T22:35:04.283121+0000","last_peered":"2024-09-10T22:35:04.283121+0000","last_clean":"2024-09-10T22:35:04.283121+0000","last_became_active":"2024-09-10T22:35:03.062374+0000","last_became_peered":"2024-09-10T22:35:03.062374+0000","last_unstale":"2024-09-10T22:35:04.283121+0000","last_undegraded":"2024-09-10T22:35:04.283121+0000","last_fullsized":"2024-09-10T22:35:04.283121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:26.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062695+0000","last_change":"2024-09-10T22:35:02.059477+0000","last_active":"2024-09-10T22:35:04.062695+0000","last_peered":"2024-09-10T22:35:04.062695+0000","last_clean":"2024-09-10T22:35:04.062695+0000","last_became_active":"2024-09-10T22:35:02.058863+0000","last_became_peered":"2024-09-10T22:35:02.058863+0000","last_unstale":"2024-09-10T22:35:04.062695+0000","last_undegraded":"2024-09-10T22:35:04.062695+0000","last_fullsized":"2024-09-10T22:35:04.062695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:44.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284043+0000","last_change":"2024-09-10T22:35:03.073670+0000","last_active":"2024-09-10T22:35:04.284043+0000","last_peered":"2024-09-10T22:35:04.284043+0000","last_clean":"2024-09-10T22:35:04.284043+0000","last_became_active":"2024-09-10T22:35:03.073362+0000","last_became_peered":"2024-09-10T22:35:03.073362+0000","last_unstale":"2024-09-10T22:35:04.284043+0000","last_undegraded":"2024-09-10T22:35:04.284043+0000","last_fullsized":"2024-09-10T22:35:04.284043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:56.758850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283238+0000","last_change":"2024-09-10T22:35:02.060433+0000","last_active":"2024-09-10T22:35:04.283238+0000","last_peered":"2024-09-10T22:35:04.283238+0000","last_clean":"2024-09-10T22:35:04.283238+0000","last_became_active":"2024-09-10T22:35:02.059798+0000","last_became_peered":"2024-09-10T22:35:02.059798+0000","last_unstale":"2024-09-10T22:35:04.283238+0000","last_undegraded":"2024-09-10T22:35:04.283238+0000","last_fullsized":"2024-09-10T22:35:04.283238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:07.281434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283176+0000","last_change":"2024-09-10T22:35:03.065328+0000","last_active":"2024-09-10T22:35:04.283176+0000","last_peered":"2024-09-10T22:35:04.283176+0000","last_clean":"2024-09-10T22:35:04.283176+0000","last_became_active":"2024-09-10T22:35:03.065050+0000","last_became_peered":"2024-09-10T22:35:03.065050+0000","last_unstale":"2024-09-10T22:35:04.283176+0000","last_undegraded":"2024-09-10T22:35:04.283176+0000","last_fullsized":"2024-09-10T22:35:04.283176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:28.267375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282601+0000","last_change":"2024-09-10T22:35:02.058914+0000","last_active":"2024-09-10T22:35:04.282601+0000","last_peered":"2024-09-10T22:35:04.282601+0000","last_clean":"2024-09-10T22:35:04.282601+0000","last_became_active":"2024-09-10T22:35:02.058054+0000","last_became_peered":"2024-09-10T22:35:02.058054+0000","last_unstale":"2024-09-10T22:35:04.282601+0000","last_undegraded":"2024-09-10T22:35:04.282601+0000","last_fullsized":"2024-09-10T22:35:04.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:07.958631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283220+0000","last_change":"2024-09-10T22:35:03.074096+0000","last_active":"2024-09-10T22:35:04.283220+0000","last_peered":"2024-09-10T22:35:04.283220+0000","last_clean":"2024-09-10T22:35:04.283220+0000","last_became_active":"2024-09-10T22:35:03.073454+0000","last_became_peered":"2024-09-10T22:35:03.073454+0000","last_unstale":"2024-09-10T22:35:04.283220+0000","last_undegraded":"2024-09-10T22:35:04.283220+0000","last_fullsized":"2024-09-10T22:35:04.283220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:16.644778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063831+0000","last_change":"2024-09-10T22:35:02.059067+0000","last_active":"2024-09-10T22:35:04.063831+0000","last_peered":"2024-09-10T22:35:04.063831+0000","last_clean":"2024-09-10T22:35:04.063831+0000","last_became_active":"2024-09-10T22:35:02.058800+0000","last_became_peered":"2024-09-10T22:35:02.058800+0000","last_unstale":"2024-09-10T22:35:04.063831+0000","last_undegraded":"2024-09-10T22:35:04.063831+0000","last_fullsized":"2024-09-10T22:35:04.063831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:32.586124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283832+0000","last_change":"2024-09-10T22:35:03.066866+0000","last_active":"2024-09-10T22:35:04.283832+0000","last_peered":"2024-09-10T22:35:04.283832+0000","last_clean":"2024-09-10T22:35:04.283832+0000","last_became_active":"2024-09-10T22:35:03.066570+0000","last_became_peered":"2024-09-10T22:35:03.066570+0000","last_unstale":"2024-09-10T22:35:04.283832+0000","last_undegraded":"2024-09-10T22:35:04.283832+0000","last_fullsized":"2024-09-10T22:35:04.283832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:57.797898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282840+0000","last_change":"2024-09-10T22:35:02.056593+0000","last_active":"2024-09-10T22:35:04.282840+0000","last_peered":"2024-09-10T22:35:04.282840+0000","last_clean":"2024-09-10T22:35:04.282840+0000","last_became_active":"2024-09-10T22:35:02.056253+0000","last_became_peered":"2024-09-10T22:35:02.056253+0000","last_unstale":"2024-09-10T22:35:04.282840+0000","last_undegraded":"2024-09-10T22:35:04.282840+0000","last_fullsized":"2024-09-10T22:35:04.282840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284206+0000","last_change":"2024-09-10T22:35:03.073877+0000","last_active":"2024-09-10T22:35:04.284206+0000","last_peered":"2024-09-10T22:35:04.284206+0000","last_clean":"2024-09-10T22:35:04.284206+0000","last_became_active":"2024-09-10T22:35:03.073479+0000","last_became_peered":"2024-09-10T22:35:03.073479+0000","last_unstale":"2024-09-10T22:35:04.284206+0000","last_undegraded":"2024-09-10T22:35:04.284206+0000","last_fullsized":"2024-09-10T22:35:04.284206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:31.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060976+0000","last_change":"2024-09-10T22:35:02.059186+0000","last_active":"2024-09-10T22:35:04.060976+0000","last_peered":"2024-09-10T22:35:04.060976+0000","last_clean":"2024-09-10T22:35:04.060976+0000","last_became_active":"2024-09-10T22:35:02.058905+0000","last_became_peered":"2024-09-10T22:35:02.058905+0000","last_unstale":"2024-09-10T22:35:04.060976+0000","last_undegraded":"2024-09-10T22:35:04.060976+0000","last_fullsized":"2024-09-10T22:35:04.060976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062968+0000","last_change":"2024-09-10T22:35:03.076118+0000","last_active":"2024-09-10T22:35:04.062968+0000","last_peered":"2024-09-10T22:35:04.062968+0000","last_clean":"2024-09-10T22:35:04.062968+0000","last_became_active":"2024-09-10T22:35:03.075621+0000","last_became_peered":"2024-09-10T22:35:03.075621+0000","last_unstale":"2024-09-10T22:35:04.062968+0000","last_undegraded":"2024-09-10T22:35:04.062968+0000","last_fullsized":"2024-09-10T22:35:04.062968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:48.372055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283210+0000","last_change":"2024-09-10T22:35:02.059788+0000","last_active":"2024-09-10T22:35:04.283210+0000","last_peered":"2024-09-10T22:35:04.283210+0000","last_clean":"2024-09-10T22:35:04.283210+0000","last_became_active":"2024-09-10T22:35:02.059419+0000","last_became_peered":"2024-09-10T22:35:02.059419+0000","last_unstale":"2024-09-10T22:35:04.283210+0000","last_undegraded":"2024-09-10T22:35:04.283210+0000","last_fullsized":"2024-09-10T22:35:04.283210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283722+0000","last_change":"2024-09-10T22:35:03.069011+0000","last_active":"2024-09-10T22:35:04.283722+0000","last_peered":"2024-09-10T22:35:04.283722+0000","last_clean":"2024-09-10T22:35:04.283722+0000","last_became_active":"2024-09-10T22:35:03.068736+0000","last_became_peered":"2024-09-10T22:35:03.068736+0000","last_unstale":"2024-09-10T22:35:04.283722+0000","last_undegraded":"2024-09-10T22:35:04.283722+0000","last_fullsized":"2024-09-10T22:35:04.283722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:28.595416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282999+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:04.282999+0000","last_peered":"2024-09-10T22:35:04.282999+0000","last_clean":"2024-09-10T22:35:04.282999+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:04.282999+0000","last_undegraded":"2024-09-10T22:35:04.282999+0000","last_fullsized":"2024-09-10T22:35:04.282999+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.061890+0000","last_change":"2024-09-10T22:35:02.057812+0000","last_active":"2024-09-10T22:35:04.061890+0000","last_peered":"2024-09-10T22:35:04.061890+0000","last_clean":"2024-09-10T22:35:04.061890+0000","last_became_active":"2024-09-10T22:35:02.057415+0000","last_became_peered":"2024-09-10T22:35:02.057415+0000","last_unstale":"2024-09-10T22:35:04.061890+0000","last_undegraded":"2024-09-10T22:35:04.061890+0000","last_fullsized":"2024-09-10T22:35:04.061890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:28.670744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283212+0000","last_change":"2024-09-10T22:35:03.079281+0000","last_active":"2024-09-10T22:35:04.283212+0000","last_peered":"2024-09-10T22:35:04.283212+0000","last_clean":"2024-09-10T22:35:04.283212+0000","last_became_active":"2024-09-10T22:35:03.078916+0000","last_became_peered":"2024-09-10T22:35:03.078916+0000","last_unstale":"2024-09-10T22:35:04.283212+0000","last_undegraded":"2024-09-10T22:35:04.283212+0000","last_fullsized":"2024-09-10T22:35:04.283212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282681+0000","last_change":"2024-09-10T22:35:02.065966+0000","last_active":"2024-09-10T22:35:04.282681+0000","last_peered":"2024-09-10T22:35:04.282681+0000","last_clean":"2024-09-10T22:35:04.282681+0000","last_became_active":"2024-09-10T22:35:02.065658+0000","last_became_peered":"2024-09-10T22:35:02.065658+0000","last_unstale":"2024-09-10T22:35:04.282681+0000","last_undegraded":"2024-09-10T22:35:04.282681+0000","last_fullsized":"2024-09-10T22:35:04.282681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:39.077105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282773+0000","last_change":"2024-09-10T22:35:03.055020+0000","last_active":"2024-09-10T22:35:04.282773+0000","last_peered":"2024-09-10T22:35:04.282773+0000","last_clean":"2024-09-10T22:35:04.282773+0000","last_became_active":"2024-09-10T22:35:03.052854+0000","last_became_peered":"2024-09-10T22:35:03.052854+0000","last_unstale":"2024-09-10T22:35:04.282773+0000","last_undegraded":"2024-09-10T22:35:04.282773+0000","last_fullsized":"2024-09-10T22:35:04.282773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:40.082137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283032+0000","last_change":"2024-09-10T22:35:02.057593+0000","last_active":"2024-09-10T22:35:04.283032+0000","last_peered":"2024-09-10T22:35:04.283032+0000","last_clean":"2024-09-10T22:35:04.283032+0000","last_became_active":"2024-09-10T22:35:02.057203+0000","last_became_peered":"2024-09-10T22:35:02.057203+0000","last_unstale":"2024-09-10T22:35:04.283032+0000","last_undegraded":"2024-09-10T22:35:04.283032+0000","last_fullsized":"2024-09-10T22:35:04.283032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:49.760894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060660+0000","last_change":"2024-09-10T22:35:03.057957+0000","last_active":"2024-09-10T22:35:04.060660+0000","last_peered":"2024-09-10T22:35:04.060660+0000","last_clean":"2024-09-10T22:35:04.060660+0000","last_became_active":"2024-09-10T22:35:03.057658+0000","last_became_peered":"2024-09-10T22:35:03.057658+0000","last_unstale":"2024-09-10T22:35:04.060660+0000","last_undegraded":"2024-09-10T22:35:04.060660+0000","last_fullsized":"2024-09-10T22:35:04.060660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062941+0000","last_change":"2024-09-10T22:35:02.059793+0000","last_active":"2024-09-10T22:35:04.062941+0000","last_peered":"2024-09-10T22:35:04.062941+0000","last_clean":"2024-09-10T22:35:04.062941+0000","last_became_active":"2024-09-10T22:35:02.059208+0000","last_became_peered":"2024-09-10T22:35:02.059208+0000","last_unstale":"2024-09-10T22:35:04.062941+0000","last_undegraded":"2024-09-10T22:35:04.062941+0000","last_fullsized":"2024-09-10T22:35:04.062941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:21.383738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063522+0000","last_change":"2024-09-10T22:35:03.067755+0000","last_active":"2024-09-10T22:35:04.063522+0000","last_peered":"2024-09-10T22:35:04.063522+0000","last_clean":"2024-09-10T22:35:04.063522+0000","last_became_active":"2024-09-10T22:35:03.067489+0000","last_became_peered":"2024-09-10T22:35:03.067489+0000","last_unstale":"2024-09-10T22:35:04.063522+0000","last_undegraded":"2024-09-10T22:35:04.063522+0000","last_fullsized":"2024-09-10T22:35:04.063522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:22.388769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060864+0000","last_change":"2024-09-10T22:35:02.063524+0000","last_active":"2024-09-10T22:35:04.060864+0000","last_peered":"2024-09-10T22:35:04.060864+0000","last_clean":"2024-09-10T22:35:04.060864+0000","last_became_active":"2024-09-10T22:35:02.063232+0000","last_became_peered":"2024-09-10T22:35:02.063232+0000","last_unstale":"2024-09-10T22:35:04.060864+0000","last_undegraded":"2024-09-10T22:35:04.060864+0000","last_fullsized":"2024-09-10T22:35:04.060864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062991+0000","last_change":"2024-09-10T22:35:03.071953+0000","last_active":"2024-09-10T22:35:04.062991+0000","last_peered":"2024-09-10T22:35:04.062991+0000","last_clean":"2024-09-10T22:35:04.062991+0000","last_became_active":"2024-09-10T22:35:03.071660+0000","last_became_peered":"2024-09-10T22:35:03.071660+0000","last_unstale":"2024-09-10T22:35:04.062991+0000","last_undegraded":"2024-09-10T22:35:04.062991+0000","last_fullsized":"2024-09-10T22:35:04.062991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:15.659782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284091+0000","last_change":"2024-09-10T22:35:02.067388+0000","last_active":"2024-09-10T22:35:04.284091+0000","last_peered":"2024-09-10T22:35:04.284091+0000","last_clean":"2024-09-10T22:35:04.284091+0000","last_became_active":"2024-09-10T22:35:02.067088+0000","last_became_peered":"2024-09-10T22:35:02.067088+0000","last_unstale":"2024-09-10T22:35:04.284091+0000","last_undegraded":"2024-09-10T22:35:04.284091+0000","last_fullsized":"2024-09-10T22:35:04.284091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:36.542782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060999+0000","last_change":"2024-09-10T22:35:03.058800+0000","last_active":"2024-09-10T22:35:04.060999+0000","last_peered":"2024-09-10T22:35:04.060999+0000","last_clean":"2024-09-10T22:35:04.060999+0000","last_became_active":"2024-09-10T22:35:03.058503+0000","last_became_peered":"2024-09-10T22:35:03.058503+0000","last_unstale":"2024-09-10T22:35:04.060999+0000","last_undegraded":"2024-09-10T22:35:04.060999+0000","last_fullsized":"2024-09-10T22:35:04.060999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.058112+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.057838+0000","last_became_peered":"2024-09-10T22:35:02.057838+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:20.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283672+0000","last_change":"2024-09-10T22:35:03.072054+0000","last_active":"2024-09-10T22:35:04.283672+0000","last_peered":"2024-09-10T22:35:04.283672+0000","last_clean":"2024-09-10T22:35:04.283672+0000","last_became_active":"2024-09-10T22:35:03.071739+0000","last_became_peered":"2024-09-10T22:35:03.071739+0000","last_unstale":"2024-09-10T22:35:04.283672+0000","last_undegraded":"2024-09-10T22:35:04.283672+0000","last_fullsized":"2024-09-10T22:35:04.283672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283676+0000","last_change":"2024-09-10T22:35:02.064151+0000","last_active":"2024-09-10T22:35:04.283676+0000","last_peered":"2024-09-10T22:35:04.283676+0000","last_clean":"2024-09-10T22:35:04.283676+0000","last_became_active":"2024-09-10T22:35:02.063573+0000","last_became_peered":"2024-09-10T22:35:02.063573+0000","last_unstale":"2024-09-10T22:35:04.283676+0000","last_undegraded":"2024-09-10T22:35:04.283676+0000","last_fullsized":"2024-09-10T22:35:04.283676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:21.244402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283654+0000","last_change":"2024-09-10T22:35:03.057069+0000","last_active":"2024-09-10T22:35:04.283654+0000","last_peered":"2024-09-10T22:35:04.283654+0000","last_clean":"2024-09-10T22:35:04.283654+0000","last_became_active":"2024-09-10T22:35:03.056760+0000","last_became_peered":"2024-09-10T22:35:03.056760+0000","last_unstale":"2024-09-10T22:35:04.283654+0000","last_undegraded":"2024-09-10T22:35:04.283654+0000","last_fullsized":"2024-09-10T22:35:04.283654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:11.516935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282652+0000","last_change":"2024-09-10T22:35:02.066275+0000","last_active":"2024-09-10T22:35:04.282652+0000","last_peered":"2024-09-10T22:35:04.282652+0000","last_clean":"2024-09-10T22:35:04.282652+0000","last_became_active":"2024-09-10T22:35:02.065887+0000","last_became_peered":"2024-09-10T22:35:02.065887+0000","last_unstale":"2024-09-10T22:35:04.282652+0000","last_undegraded":"2024-09-10T22:35:04.282652+0000","last_fullsized":"2024-09-10T22:35:04.282652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:09.305553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062290+0000","last_change":"2024-09-10T22:35:03.054840+0000","last_active":"2024-09-10T22:35:04.062290+0000","last_peered":"2024-09-10T22:35:04.062290+0000","last_clean":"2024-09-10T22:35:04.062290+0000","last_became_active":"2024-09-10T22:35:03.054562+0000","last_became_peered":"2024-09-10T22:35:03.054562+0000","last_unstale":"2024-09-10T22:35:04.062290+0000","last_undegraded":"2024-09-10T22:35:04.062290+0000","last_fullsized":"2024-09-10T22:35:04.062290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:29.675776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060764+0000","last_change":"2024-09-10T22:35:02.061273+0000","last_active":"2024-09-10T22:35:04.060764+0000","last_peered":"2024-09-10T22:35:04.060764+0000","last_clean":"2024-09-10T22:35:04.060764+0000","last_became_active":"2024-09-10T22:35:02.060953+0000","last_became_peered":"2024-09-10T22:35:02.060953+0000","last_unstale":"2024-09-10T22:35:04.060764+0000","last_undegraded":"2024-09-10T22:35:04.060764+0000","last_fullsized":"2024-09-10T22:35:04.060764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:32.172943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284309+0000","last_change":"2024-09-10T22:35:03.078424+0000","last_active":"2024-09-10T22:35:04.284309+0000","last_peered":"2024-09-10T22:35:04.284309+0000","last_clean":"2024-09-10T22:35:04.284309+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.284309+0000","last_undegraded":"2024-09-10T22:35:04.284309+0000","last_fullsized":"2024-09-10T22:35:04.284309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:48.985804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.061749+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.061248+0000","last_became_peered":"2024-09-10T22:35:02.061248+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:49.960770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283758+0000","last_change":"2024-09-10T22:35:03.076736+0000","last_active":"2024-09-10T22:35:04.283758+0000","last_peered":"2024-09-10T22:35:04.283758+0000","last_clean":"2024-09-10T22:35:04.283758+0000","last_became_active":"2024-09-10T22:35:03.075379+0000","last_became_peered":"2024-09-10T22:35:03.075379+0000","last_unstale":"2024-09-10T22:35:04.283758+0000","last_undegraded":"2024-09-10T22:35:04.283758+0000","last_fullsized":"2024-09-10T22:35:04.283758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.399411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284154+0000","last_change":"2024-09-10T22:35:02.065909+0000","last_active":"2024-09-10T22:35:04.284154+0000","last_peered":"2024-09-10T22:35:04.284154+0000","last_clean":"2024-09-10T22:35:04.284154+0000","last_became_active":"2024-09-10T22:35:02.065539+0000","last_became_peered":"2024-09-10T22:35:02.065539+0000","last_unstale":"2024-09-10T22:35:04.284154+0000","last_undegraded":"2024-09-10T22:35:04.284154+0000","last_fullsized":"2024-09-10T22:35:04.284154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:32.374399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283186+0000","last_change":"2024-09-10T22:35:03.060477+0000","last_active":"2024-09-10T22:35:04.283186+0000","last_peered":"2024-09-10T22:35:04.283186+0000","last_clean":"2024-09-10T22:35:04.283186+0000","last_became_active":"2024-09-10T22:35:03.060190+0000","last_became_peered":"2024-09-10T22:35:03.060190+0000","last_unstale":"2024-09-10T22:35:04.283186+0000","last_undegraded":"2024-09-10T22:35:04.283186+0000","last_fullsized":"2024-09-10T22:35:04.283186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:56.641563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062898+0000","last_change":"2024-09-10T22:35:03.060225+0000","last_active":"2024-09-10T22:35:04.062898+0000","last_peered":"2024-09-10T22:35:04.062898+0000","last_clean":"2024-09-10T22:35:04.062898+0000","last_became_active":"2024-09-10T22:35:03.059807+0000","last_became_peered":"2024-09-10T22:35:03.059807+0000","last_unstale":"2024-09-10T22:35:04.062898+0000","last_undegraded":"2024-09-10T22:35:04.062898+0000","last_fullsized":"2024-09-10T22:35:04.062898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:12.132002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283516+0000","last_change":"2024-09-10T22:35:02.066541+0000","last_active":"2024-09-10T22:35:04.283516+0000","last_peered":"2024-09-10T22:35:04.283516+0000","last_clean":"2024-09-10T22:35:04.283516+0000","last_became_active":"2024-09-10T22:35:02.066259+0000","last_became_peered":"2024-09-10T22:35:02.066259+0000","last_unstale":"2024-09-10T22:35:04.283516+0000","last_undegraded":"2024-09-10T22:35:04.283516+0000","last_fullsized":"2024-09-10T22:35:04.283516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:58.279821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062332+0000","last_change":"2024-09-10T22:35:03.068676+0000","last_active":"2024-09-10T22:35:04.062332+0000","last_peered":"2024-09-10T22:35:04.062332+0000","last_clean":"2024-09-10T22:35:04.062332+0000","last_became_active":"2024-09-10T22:35:03.068384+0000","last_became_peered":"2024-09-10T22:35:03.068384+0000","last_unstale":"2024-09-10T22:35:04.062332+0000","last_undegraded":"2024-09-10T22:35:04.062332+0000","last_fullsized":"2024-09-10T22:35:04.062332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:56.275057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283838+0000","last_change":"2024-09-10T22:35:02.065459+0000","last_active":"2024-09-10T22:35:04.283838+0000","last_peered":"2024-09-10T22:35:04.283838+0000","last_clean":"2024-09-10T22:35:04.283838+0000","last_became_active":"2024-09-10T22:35:02.065145+0000","last_became_peered":"2024-09-10T22:35:02.065145+0000","last_unstale":"2024-09-10T22:35:04.283838+0000","last_undegraded":"2024-09-10T22:35:04.283838+0000","last_fullsized":"2024-09-10T22:35:04.283838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:21.383738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283725+0000","last_change":"2024-09-10T22:35:03.063955+0000","last_active":"2024-09-10T22:35:04.283725+0000","last_peered":"2024-09-10T22:35:04.283725+0000","last_clean":"2024-09-10T22:35:04.283725+0000","last_became_active":"2024-09-10T22:35:03.063679+0000","last_became_peered":"2024-09-10T22:35:03.063679+0000","last_unstale":"2024-09-10T22:35:04.283725+0000","last_undegraded":"2024-09-10T22:35:04.283725+0000","last_fullsized":"2024-09-10T22:35:04.283725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:56.641563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283509+0000","last_change":"2024-09-10T22:35:02.062827+0000","last_active":"2024-09-10T22:35:04.283509+0000","last_peered":"2024-09-10T22:35:04.283509+0000","last_clean":"2024-09-10T22:35:04.283509+0000","last_became_active":"2024-09-10T22:35:02.062342+0000","last_became_peered":"2024-09-10T22:35:02.062342+0000","last_unstale":"2024-09-10T22:35:04.283509+0000","last_undegraded":"2024-09-10T22:35:04.283509+0000","last_fullsized":"2024-09-10T22:35:04.283509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283072+0000","last_change":"2024-09-10T22:35:03.080815+0000","last_active":"2024-09-10T22:35:04.283072+0000","last_peered":"2024-09-10T22:35:04.283072+0000","last_clean":"2024-09-10T22:35:04.283072+0000","last_became_active":"2024-09-10T22:35:03.080484+0000","last_became_peered":"2024-09-10T22:35:03.080484+0000","last_unstale":"2024-09-10T22:35:04.283072+0000","last_undegraded":"2024-09-10T22:35:04.283072+0000","last_fullsized":"2024-09-10T22:35:04.283072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:58.700652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283792+0000","last_change":"2024-09-10T22:35:02.065788+0000","last_active":"2024-09-10T22:35:04.283792+0000","last_peered":"2024-09-10T22:35:04.283792+0000","last_clean":"2024-09-10T22:35:04.283792+0000","last_became_active":"2024-09-10T22:35:02.065500+0000","last_became_peered":"2024-09-10T22:35:02.065500+0000","last_unstale":"2024-09-10T22:35:04.283792+0000","last_undegraded":"2024-09-10T22:35:04.283792+0000","last_fullsized":"2024-09-10T22:35:04.283792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:10.573361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":663271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":72411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":72447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":815104,"data_stored":663327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:11.856 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:35:11.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T22:35:11.857 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T22:35:11.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.997+0000 7faaa3394640 1 -- 172.21.15.66:0/2821637703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa880063f0 msgr2=0x7faa8800cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.997+0000 7faaa3394640 1 --2- 172.21.15.66:0/2821637703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa880063f0 0x7faa8800cad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa94005fd0 tx=0x7faa88006940 comp rx=0 tx=0).stop 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.997+0000 7faaa3394640 1 -- 172.21.15.66:0/2821637703 shutdown_connections 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.997+0000 7faaa3394640 1 --2- 172.21.15.66:0/2821637703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa880063f0 0x7faa8800cad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.997+0000 7faaa3394640 1 -- 172.21.15.66:0/2821637703 >> 172.21.15.66:0/2821637703 conn(0x7faa9c0b1660 msgr2=0x7faa9c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.997+0000 7faaa3394640 1 -- 172.21.15.66:0/2821637703 shutdown_connections 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.997+0000 7faaa3394640 1 -- 172.21.15.66:0/2821637703 wait complete. 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.998+0000 7faaa3394640 1 Processor -- start 2024-09-10T22:35:11.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.998+0000 7faaa3394640 1 -- start start 2024-09-10T22:35:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa3394640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880063f0 0x7faa9c0c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa3394640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9c0ae120 0x7faa9c1f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa3394640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa9c0c98e0 0x7faa9c15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa3394640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c06f430 con 0x7faa9c0ae120 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa3394640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c0ca640 con 0x7faa880063f0 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa3394640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7faa9c06f5b0 con 0x7faa9c0c98e0 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa092f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9c0ae120 0x7faa9c1f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa092f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9c0ae120 0x7faa9c1f37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55784/0 (socket says 172.21.15.66:55784) 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa092f640 1 -- 172.21.15.66:0/915155545 learned_addr learned my addr 172.21.15.66:0/915155545 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa1130640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880063f0 0x7faa9c0c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:11.999+0000 7faaa1931640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa9c0c98e0 0x7faa9c15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa092f640 1 -- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa9c0c98e0 msgr2=0x7faa9c15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa092f640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa9c0c98e0 0x7faa9c15e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa092f640 1 -- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880063f0 msgr2=0x7faa9c0c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa092f640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880063f0 0x7faa9c0c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa092f640 1 -- 172.21.15.66:0/915155545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c0c8880 con 0x7faa9c0ae120 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa092f640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9c0ae120 0x7faa9c1f37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa900066b0 tx=0x7faa900039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:12.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faa8dffb640 1 -- 172.21.15.66:0/915155545 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90015070 con 0x7faa9c0ae120 2024-09-10T22:35:12.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faa8dffb640 1 -- 172.21.15.66:0/915155545 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa90011900 con 0x7faa9c0ae120 2024-09-10T22:35:12.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c0c9120 con 0x7faa9c0ae120 2024-09-10T22:35:12.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faa8dffb640 1 -- 172.21.15.66:0/915155545 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90011d30 con 0x7faa9c0ae120 2024-09-10T22:35:12.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.000+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c1f2ff0 con 0x7faa9c0ae120 2024-09-10T22:35:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.001+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c0cd390 con 0x7faa9c0ae120 2024-09-10T22:35:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.002+0000 7faa8dffb640 1 -- 172.21.15.66:0/915155545 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7faa90010420 con 0x7faa9c0ae120 2024-09-10T22:35:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.002+0000 7faa8dffb640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7faa80077fa0 0x7faa8007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.002+0000 7faa8dffb640 1 -- 172.21.15.66:0/915155545 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faa9009acc0 con 0x7faa9c0ae120 2024-09-10T22:35:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.002+0000 7faaa1130640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7faa80077fa0 0x7faa8007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.003+0000 7faaa1130640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7faa80077fa0 0x7faa8007a460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa98000c00 tx=0x7faa98006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:12.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.006+0000 7faa8dffb640 1 -- 172.21.15.66:0/915155545 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7faa90063240 con 0x7faa9c0ae120 2024-09-10T22:35:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.191+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7faa9c0ced80 con 0x7faa9c0ae120 2024-09-10T22:35:12.192 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.191+0000 7faa8dffb640 1 -- 172.21.15.66:0/915155545 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7faa90067c70 con 0x7faa9c0ae120 2024-09-10T22:35:12.192 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7faa80077fa0 msgr2=0x7faa8007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7faa80077fa0 0x7faa8007a460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa98000c00 tx=0x7faa98006b70 comp rx=0 tx=0).stop 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9c0ae120 msgr2=0x7faa9c1f37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9c0ae120 0x7faa9c1f37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa900066b0 tx=0x7faa900039c0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 shutdown_connections 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7faa80077fa0 0x7faa8007a460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7faa9c0c98e0 0x7faa9c15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7faa9c0ae120 0x7faa9c1f37f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 --2- 172.21.15.66:0/915155545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa880063f0 0x7faa9c0c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 >> 172.21.15.66:0/915155545 conn(0x7faa9c0b1660 msgr2=0x7faa9c0b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:12.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 shutdown_connections 2024-09-10T22:35:12.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.193+0000 7faaa3394640 1 -- 172.21.15.66:0/915155545 wait complete. 2024-09-10T22:35:12.205 INFO:teuthology.orchestra.run.smithi066.stdout:[{"version":27,"timestamp":"2024-09-10T22:34:50.299129+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"5031.347720"}]}] 2024-09-10T22:35:12.205 INFO:tasks.ceph_manager:config epoch is 27 2024-09-10T22:35:12.206 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T22:35:12.206 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T22:35:12.206 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T22:35:12.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 -- 172.21.15.66:0/4282221723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec15e060 msgr2=0x7f65ec0d4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 --2- 172.21.15.66:0/4282221723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec15e060 0x7f65ec0d4680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0d26d0 tx=0x7f65dc02a460 comp rx=0 tx=0).stop 2024-09-10T22:35:12.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 -- 172.21.15.66:0/4282221723 shutdown_connections 2024-09-10T22:35:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 --2- 172.21.15.66:0/4282221723 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65ec0d0d80 0x7f65ec0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 --2- 172.21.15.66:0/4282221723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec15e060 0x7f65ec0d4680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 -- 172.21.15.66:0/4282221723 >> 172.21.15.66:0/4282221723 conn(0x7f65ec09d270 msgr2=0x7f65ec0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 -- 172.21.15.66:0/4282221723 shutdown_connections 2024-09-10T22:35:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.310+0000 7f65f1665640 1 -- 172.21.15.66:0/4282221723 wait complete. 2024-09-10T22:35:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.311+0000 7f65f1665640 1 Processor -- start 2024-09-10T22:35:12.312 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.311+0000 7f65f1665640 1 -- start start 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65f1665640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec0b3060 0x7f65ec160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65f1665640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65ec0d0d80 0x7f65ec15f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65f1665640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65ec15e060 0x7f65ec0b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65f1665640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec0b4360 con 0x7f65ec0b3060 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65f1665640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec0b41e0 con 0x7f65ec0d0d80 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65f1665640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f65ec0b44e0 con 0x7f65ec15e060 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec0b3060 0x7f65ec160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec0b3060 0x7f65ec160cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55808/0 (socket says 172.21.15.66:55808) 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 -- 172.21.15.66:0/824430786 learned_addr learned my addr 172.21.15.66:0/824430786 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65ea7fc640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65ec0d0d80 0x7f65ec15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.315 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eb7fe640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65ec15e060 0x7f65ec0b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 -- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65ec15e060 msgr2=0x7f65ec0b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65ec15e060 0x7f65ec0b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 -- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65ec0d0d80 msgr2=0x7f65ec15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65ec0d0d80 0x7f65ec15f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 -- 172.21.15.66:0/824430786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec0b9df0 con 0x7f65ec0b3060 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.312+0000 7f65eaffd640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec0b3060 0x7f65ec160cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005fd0 tx=0x7f65dc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.313+0000 7f65d37fe640 1 -- 172.21.15.66:0/824430786 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc03e030 con 0x7f65ec0b3060 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.313+0000 7f65d37fe640 1 -- 172.21.15.66:0/824430786 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65dc0047a0 con 0x7f65ec0b3060 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.313+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec1612a0 con 0x7f65ec0b3060 2024-09-10T22:35:12.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.313+0000 7f65d37fe640 1 -- 172.21.15.66:0/824430786 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc035040 con 0x7f65ec0b3060 2024-09-10T22:35:12.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.313+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec15e9b0 con 0x7f65ec0b3060 2024-09-10T22:35:12.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.314+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec0b4a40 con 0x7f65ec0b3060 2024-09-10T22:35:12.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.314+0000 7f65d37fe640 1 -- 172.21.15.66:0/824430786 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f65dc004bf0 con 0x7f65ec0b3060 2024-09-10T22:35:12.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.315+0000 7f65d37fe640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f65d8077f70 0x7f65d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.315+0000 7f65d37fe640 1 -- 172.21.15.66:0/824430786 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f65dc0ba6e0 con 0x7f65ec0b3060 2024-09-10T22:35:12.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.315+0000 7f65ea7fc640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f65d8077f70 0x7f65d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.316+0000 7f65ea7fc640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f65d8077f70 0x7f65d807a430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65ec15c410 tx=0x7f65e0007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:12.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.319+0000 7f65d37fe640 1 -- 172.21.15.66:0/824430786 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f65dc082b90 con 0x7f65ec0b3060 2024-09-10T22:35:12.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.555+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f65ec0d1900 con 0x7f65ec0b3060 2024-09-10T22:35:12.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.558+0000 7f65d37fe640 1 -- 172.21.15.66:0/824430786 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+193240 (secure 0 0 0) 0x7f65dc0875c0 con 0x7f65ec0b3060 2024-09-10T22:35:12.562 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.560+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f65d8077f70 msgr2=0x7f65d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f65d8077f70 0x7f65d807a430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65ec15c410 tx=0x7f65e0007e30 comp rx=0 tx=0).stop 2024-09-10T22:35:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec0b3060 msgr2=0x7f65ec160cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec0b3060 0x7f65ec160cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005fd0 tx=0x7f65dc0039e0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 shutdown_connections 2024-09-10T22:35:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f65d8077f70 0x7f65d807a430 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65ec15e060 0x7f65ec0b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65ec0d0d80 0x7f65ec15f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 --2- 172.21.15.66:0/824430786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec0b3060 0x7f65ec160cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 >> 172.21.15.66:0/824430786 conn(0x7f65ec09d270 msgr2=0x7f65ec0b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 shutdown_connections 2024-09-10T22:35:12.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.561+0000 7f65f1665640 1 -- 172.21.15.66:0/824430786 wait complete. 2024-09-10T22:35:12.573 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":718501963},{"type":"v1","addr":"172.21.15.94:6801","nonce":718501963}]},"active_addr":"172.21.15.94:6801/718501963","active_change":"2024-09-10T22:34:42.585003+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4109,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3030576760}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2156124737}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2004274225}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2576929019}]}]} 2024-09-10T22:35:12.577 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T22:35:12.577 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T22:35:12.577 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.682+0000 7fbda3b32640 1 -- 172.21.15.66:0/1039772754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd84004800 msgr2=0x7fbd84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.682+0000 7fbda3b32640 1 --2- 172.21.15.66:0/1039772754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd84004800 0x7fbd84004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0d0230 tx=0x7fbd84005830 comp rx=0 tx=0).stop 2024-09-10T22:35:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.682+0000 7fbda3b32640 1 -- 172.21.15.66:0/1039772754 shutdown_connections 2024-09-10T22:35:12.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.682+0000 7fbda3b32640 1 --2- 172.21.15.66:0/1039772754 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd84004800 0x7fbd84004c80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.682+0000 7fbda3b32640 1 -- 172.21.15.66:0/1039772754 >> 172.21.15.66:0/1039772754 conn(0x7fbd9c0d1d20 msgr2=0x7fbd9c0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.682+0000 7fbda3b32640 1 -- 172.21.15.66:0/1039772754 shutdown_connections 2024-09-10T22:35:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.682+0000 7fbda3b32640 1 -- 172.21.15.66:0/1039772754 wait complete. 2024-09-10T22:35:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.683+0000 7fbda3b32640 1 Processor -- start 2024-09-10T22:35:12.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.683+0000 7fbda3b32640 1 -- start start 2024-09-10T22:35:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda3b32640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd84004800 0x7fbd9c0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda3b32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd9c149dd0 0x7fbd9c1f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda3b32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c1f6910 0x7fbd9c071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda3b32640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0ce3c0 con 0x7fbd9c1f6910 2024-09-10T22:35:12.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda3b32640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0cf640 con 0x7fbd9c149dd0 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda3b32640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0ce540 con 0x7fbd84004800 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c1f6910 0x7fbd9c071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c1f6910 0x7fbd9c071950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55836/0 (socket says 172.21.15.66:55836) 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 -- 172.21.15.66:0/2914888338 learned_addr learned my addr 172.21.15.66:0/2914888338 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda10cd640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd9c149dd0 0x7fbd9c1f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda18ce640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd84004800 0x7fbd9c0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 -- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd84004800 msgr2=0x7fbd9c0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd84004800 0x7fbd9c0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 -- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd9c149dd0 msgr2=0x7fbd9c1f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd9c149dd0 0x7fbd9c1f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 -- 172.21.15.66:0/2914888338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c1f6e00 con 0x7fbd9c1f6910 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.684+0000 7fbda20cf640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c1f6910 0x7fbd9c071950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0cc430 tx=0x7fbd8c006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.685+0000 7fbd9a7fc640 1 -- 172.21.15.66:0/2914888338 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00fa20 con 0x7fbd9c1f6910 2024-09-10T22:35:12.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.685+0000 7fbd9a7fc640 1 -- 172.21.15.66:0/2914888338 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd8c018070 con 0x7fbd9c1f6910 2024-09-10T22:35:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.685+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c1f70e0 con 0x7fbd9c1f6910 2024-09-10T22:35:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.685+0000 7fbd9a7fc640 1 -- 172.21.15.66:0/2914888338 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00fe80 con 0x7fbd9c1f6910 2024-09-10T22:35:12.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.685+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c0b68e0 con 0x7fbd9c1f6910 2024-09-10T22:35:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.686+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c1f7320 con 0x7fbd9c1f6910 2024-09-10T22:35:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.686+0000 7fbd9a7fc640 1 -- 172.21.15.66:0/2914888338 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbd8c01d460 con 0x7fbd9c1f6910 2024-09-10T22:35:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.687+0000 7fbd9a7fc640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbd88077f70 0x7fbd8807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.687+0000 7fbd9a7fc640 1 -- 172.21.15.66:0/2914888338 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd8c09a450 con 0x7fbd9c1f6910 2024-09-10T22:35:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.687+0000 7fbda18ce640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbd88077f70 0x7fbd8807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:12.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.687+0000 7fbda18ce640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbd88077f70 0x7fbd8807a430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009c50 tx=0x7fbd94005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.691+0000 7fbd9a7fc640 1 -- 172.21.15.66:0/2914888338 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbd9c1f7320 con 0x7fbd9c1f6910 2024-09-10T22:35:12.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.877+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd9c0ceba0 con 0x7fbd9c1f6910 2024-09-10T22:35:12.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.878+0000 7fbd9a7fc640 1 -- 172.21.15.66:0/2914888338 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16641 (secure 0 0 0) 0x7fbd9c0ceba0 con 0x7fbd9c1f6910 2024-09-10T22:35:12.880 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:12.880 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:04.052417+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbd88077f70 msgr2=0x7fbd8807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbd88077f70 0x7fbd8807a430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009c50 tx=0x7fbd94005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c1f6910 msgr2=0x7fbd9c071950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c1f6910 0x7fbd9c071950 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0cc430 tx=0x7fbd8c006660 comp rx=0 tx=0).stop 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 shutdown_connections 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbd88077f70 0x7fbd8807a430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbd9c1f6910 0x7fbd9c071950 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd9c149dd0 0x7fbd9c1f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 --2- 172.21.15.66:0/2914888338 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbd84004800 0x7fbd9c0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 >> 172.21.15.66:0/2914888338 conn(0x7fbd9c0d1d20 msgr2=0x7fbd9c0ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:12.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.880+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 shutdown_connections 2024-09-10T22:35:12.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.881+0000 7fbda3b32640 1 -- 172.21.15.66:0/2914888338 wait complete. 2024-09-10T22:35:12.892 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T22:35:12.893 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.996+0000 7f5a346dd640 1 -- 172.21.15.66:0/655047527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 msgr2=0x7f5a14003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.996+0000 7f5a346dd640 1 --2- 172.21.15.66:0/655047527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a14003e00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5a24001eb0 tx=0x7f5a14005830 comp rx=0 tx=0).stop 2024-09-10T22:35:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.997+0000 7f5a346dd640 1 -- 172.21.15.66:0/655047527 shutdown_connections 2024-09-10T22:35:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.997+0000 7f5a346dd640 1 --2- 172.21.15.66:0/655047527 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a14003e00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.997+0000 7f5a346dd640 1 -- 172.21.15.66:0/655047527 >> 172.21.15.66:0/655047527 conn(0x7f5a2c09d260 msgr2=0x7f5a2c0d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.997+0000 7f5a346dd640 1 -- 172.21.15.66:0/655047527 shutdown_connections 2024-09-10T22:35:12.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.997+0000 7f5a346dd640 1 -- 172.21.15.66:0/655047527 wait complete. 2024-09-10T22:35:12.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.998+0000 7f5a346dd640 1 Processor -- start 2024-09-10T22:35:12.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.998+0000 7f5a346dd640 1 -- start start 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a2c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2c152550 0x7f5a2c1f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a2c151730 0x7f5a2c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c0b3ff0 con 0x7f5a14003a00 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c0b3e70 con 0x7f5a2c151730 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2c0b4170 con 0x7f5a2c152550 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a2c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a2c15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55858/0 (socket says 172.21.15.66:55858) 2024-09-10T22:35:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 -- 172.21.15.66:0/2487350420 learned_addr learned my addr 172.21.15.66:0/2487350420 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 -- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2c152550 msgr2=0x7f5a2c1f88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2c152550 0x7f5a2c1f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 -- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a2c151730 msgr2=0x7f5a2c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a2c151730 0x7f5a2c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 -- 172.21.15.66:0/2487350420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c1f7e00 con 0x7f5a14003a00 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a32479640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a2c15d880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c3d0 tx=0x7f5a24003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a2affd640 1 -- 172.21.15.66:0/2487350420 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24006c50 con 0x7f5a14003a00 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a2affd640 1 -- 172.21.15.66:0/2487350420 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a240014f0 con 0x7f5a14003a00 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c160870 con 0x7f5a14003a00 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a2affd640 1 -- 172.21.15.66:0/2487350420 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2402e6a0 con 0x7f5a14003a00 2024-09-10T22:35:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:12.999+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c1f80a0 con 0x7f5a14003a00 2024-09-10T22:35:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.000+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2c0b4500 con 0x7f5a14003a00 2024-09-10T22:35:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.001+0000 7f5a2affd640 1 -- 172.21.15.66:0/2487350420 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5a24040020 con 0x7f5a14003a00 2024-09-10T22:35:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.001+0000 7f5a2affd640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a18077fa0 0x7f5a1807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.002+0000 7f5a2affd640 1 -- 172.21.15.66:0/2487350420 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5a240ba480 con 0x7f5a14003a00 2024-09-10T22:35:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.002+0000 7f5a31c78640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a18077fa0 0x7f5a1807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.002+0000 7f5a31c78640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a18077fa0 0x7f5a1807a460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a20006fd0 tx=0x7f5a20006f40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.006+0000 7f5a2affd640 1 -- 172.21.15.66:0/2487350420 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5a24047060 con 0x7f5a14003a00 2024-09-10T22:35:13.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.194+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a2c08d6e0 con 0x7f5a14003a00 2024-09-10T22:35:13.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.195+0000 7f5a2affd640 1 -- 172.21.15.66:0/2487350420 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16641 (secure 0 0 0) 0x7f5a24083010 con 0x7f5a14003a00 2024-09-10T22:35:13.196 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:13.197 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:04.052417+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.197+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a18077fa0 msgr2=0x7f5a1807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.197+0000 7f5a346dd640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a18077fa0 0x7f5a1807a460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5a20006fd0 tx=0x7f5a20006f40 comp rx=0 tx=0).stop 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.197+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 msgr2=0x7f5a2c15d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.197+0000 7f5a346dd640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a2c15d880 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c3d0 tx=0x7f5a24003c30 comp rx=0 tx=0).stop 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 shutdown_connections 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a18077fa0 0x7f5a1807a460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a2c151730 0x7f5a2c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a2c152550 0x7f5a2c1f88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 --2- 172.21.15.66:0/2487350420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a14003a00 0x7f5a2c15d880 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 >> 172.21.15.66:0/2487350420 conn(0x7f5a2c09d260 msgr2=0x7f5a2c0cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 shutdown_connections 2024-09-10T22:35:13.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.198+0000 7f5a346dd640 1 -- 172.21.15.66:0/2487350420 wait complete. 2024-09-10T22:35:13.210 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T22:35:13.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T22:35:13.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T22:35:13.212 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T22:35:13.212 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T22:35:13.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T22:35:13.213 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T22:35:13.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T22:35:13.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7fa77bcbd640 1 -- 172.21.15.66:0/2925987545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7740d4ad0 msgr2=0x7fa7740b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7f8feb7fe640 1 -- 172.21.15.66:0/1849137056 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0047a0 con 0x7f8fec0704a0 2024-09-10T22:35:13.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7fa77bcbd640 1 --2- 172.21.15.66:0/2925987545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7740d4ad0 0x7fa7740b3e50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa770005fd0 tx=0x7fa77002a530 comp rx=0 tx=0).stop 2024-09-10T22:35:13.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7fa77bcbd640 1 -- 172.21.15.66:0/2925987545 shutdown_connections 2024-09-10T22:35:13.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7fa77bcbd640 1 --2- 172.21.15.66:0/2925987545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa7740d4ad0 0x7fa7740b3e50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7fa77bcbd640 1 --2- 172.21.15.66:0/2925987545 >> v2:172.21.15.66:3300/0 conn(0x7fa774161b10 0x7fa7740b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7fa77bcbd640 1 -- 172.21.15.66:0/2925987545 >> 172.21.15.66:0/2925987545 conn(0x7fa7740d3fb0 msgr2=0x7fa7740d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.394+0000 7fa77bcbd640 1 -- 172.21.15.66:0/2925987545 shutdown_connections 2024-09-10T22:35:13.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.395+0000 7f8ff2b10640 1 -- 172.21.15.66:0/1849137056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec0704a0 msgr2=0x7f8fec073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.395+0000 7f8ff2b10640 1 --2- 172.21.15.66:0/1849137056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec0704a0 0x7f8fec073ef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005fd0 tx=0x7f8fdc02a5f0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.395+0000 7f8ff2b10640 1 -- 172.21.15.66:0/1849137056 shutdown_connections 2024-09-10T22:35:13.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.395+0000 7f8ff2b10640 1 --2- 172.21.15.66:0/1849137056 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fec074430 0x7f8fec0cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.395+0000 7f8ff2b10640 1 --2- 172.21.15.66:0/1849137056 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec0704a0 0x7f8fec073ef0 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005fd0 tx=0x7f8fdc02a5f0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.395+0000 7f8ff2b10640 1 -- 172.21.15.66:0/1849137056 >> 172.21.15.66:0/1849137056 conn(0x7f8fec06cd00 msgr2=0x7f8fec06d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.395+0000 7fa77bcbd640 1 -- 172.21.15.66:0/2925987545 wait complete. 2024-09-10T22:35:13.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f10f228c640 1 -- 172.21.15.66:0/3853711564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ec1579b0 msgr2=0x7f10ec0aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50abfff640 1 -- 172.21.15.66:0/3722579128 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac004030 con 0x7f50b4151f80 2024-09-10T22:35:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f10f228c640 1 --2- 172.21.15.66:0/3853711564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ec1579b0 0x7f10ec0aec20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005fd0 tx=0x7f10dc02b7f0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 -- 172.21.15.66:0/3722579128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 msgr2=0x7f50b4073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 --2- 172.21.15.66:0/3722579128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 0x7f50b4073670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50b40d1cf0 tx=0x7f50ac02a830 comp rx=0 tx=0).stop 2024-09-10T22:35:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 -- 172.21.15.66:0/3722579128 shutdown_connections 2024-09-10T22:35:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 --2- 172.21.15.66:0/3722579128 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b40b4150 0x7f50b415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 --2- 172.21.15.66:0/3722579128 >> v2:172.21.15.66:3300/0 conn(0x7f50b4073bb0 0x7f50b40b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 --2- 172.21.15.66:0/3722579128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 0x7f50b4073670 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 -- 172.21.15.66:0/3722579128 >> 172.21.15.66:0/3722579128 conn(0x7f50b40b27c0 msgr2=0x7f50b40b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f8ff2b10640 1 -- 172.21.15.66:0/1849137056 shutdown_connections 2024-09-10T22:35:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 -- 172.21.15.66:0/3722579128 shutdown_connections 2024-09-10T22:35:13.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f8ff2b10640 1 -- 172.21.15.66:0/1849137056 wait complete. 2024-09-10T22:35:13.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f10f228c640 1 -- 172.21.15.66:0/3853711564 shutdown_connections 2024-09-10T22:35:13.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f10f228c640 1 --2- 172.21.15.66:0/3853711564 >> v2:172.21.15.94:3300/0 conn(0x7f10ec0ccc30 0x7f10ec1639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f10f228c640 1 --2- 172.21.15.66:0/3853711564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ec1579b0 0x7f10ec0aec20 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005fd0 tx=0x7f10dc02b7f0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f10f228c640 1 -- 172.21.15.66:0/3853711564 >> 172.21.15.66:0/3853711564 conn(0x7f10ec0b1660 msgr2=0x7f10ec0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.396+0000 7f50bb62a640 1 -- 172.21.15.66:0/3722579128 wait complete. 2024-09-10T22:35:13.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f10f228c640 1 -- 172.21.15.66:0/3853711564 shutdown_connections 2024-09-10T22:35:13.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f10f228c640 1 -- 172.21.15.66:0/3853711564 wait complete. 2024-09-10T22:35:13.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7fa77bcbd640 1 Processor -- start 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f98e4f52640 1 -- 172.21.15.66:0/2413641761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e00d1ab0 msgr2=0x7f98e00d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f98e4f52640 1 --2- 172.21.15.66:0/2413641761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e00d1ab0 0x7f98e00d1f60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98e00d1830 tx=0x7f98d802aa30 comp rx=0 tx=0).stop 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f98e4f52640 1 -- 172.21.15.66:0/2413641761 shutdown_connections 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f98e4f52640 1 --2- 172.21.15.66:0/2413641761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e00d1ab0 0x7f98e00d1f60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f98e4f52640 1 --2- 172.21.15.66:0/2413641761 >> v2:172.21.15.66:3300/0 conn(0x7f98e006eaf0 0x7f98e00ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f98e4f52640 1 --2- 172.21.15.66:0/2413641761 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98e015df20 0x7f98e00cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.397+0000 7f98e4f52640 1 -- 172.21.15.66:0/2413641761 >> 172.21.15.66:0/2413641761 conn(0x7f98e009aa70 msgr2=0x7f98e009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7f98e4f52640 1 -- 172.21.15.66:0/2413641761 shutdown_connections 2024-09-10T22:35:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7f98e4f52640 1 -- 172.21.15.66:0/2413641761 wait complete. 2024-09-10T22:35:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7f8ff2b10640 1 Processor -- start 2024-09-10T22:35:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7f8ff2b10640 1 -- start start 2024-09-10T22:35:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7ff9711de640 1 -- 172.21.15.66:0/1734566863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c070530 msgr2=0x7ff96c15a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7ff9711de640 1 --2- 172.21.15.66:0/1734566863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c070530 0x7ff96c15a170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff964009be0 tx=0x7ff96402aa00 comp rx=0 tx=0).stop 2024-09-10T22:35:13.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7ff9711de640 1 -- 172.21.15.66:0/1734566863 shutdown_connections 2024-09-10T22:35:13.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7ff9711de640 1 --2- 172.21.15.66:0/1734566863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c070530 0x7ff96c15a170 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff964009be0 tx=0x7ff96402aa00 comp rx=0 tx=0).stop 2024-09-10T22:35:13.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7ff9711de640 1 -- 172.21.15.66:0/1734566863 >> 172.21.15.66:0/1734566863 conn(0x7ff96c0b16d0 msgr2=0x7ff96c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7f50bb62a640 1 Processor -- start 2024-09-10T22:35:13.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.398+0000 7ff9711de640 1 -- 172.21.15.66:0/1734566863 shutdown_connections 2024-09-10T22:35:13.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.399+0000 7f50bb62a640 1 -- start start 2024-09-10T22:35:13.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.399+0000 7fa77bcbd640 1 -- start start 2024-09-10T22:35:13.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f10f228c640 1 Processor -- start 2024-09-10T22:35:13.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f10f228c640 1 -- start start 2024-09-10T22:35:13.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f50bb62a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b4073bb0 0x7f50b4156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f8ff2b10640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fec074430 0x7f8fec144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3066398941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15b4003a70 msgr2=0x7f15b4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f15d1f6b640 1 --2- 172.21.15.66:0/3066398941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15b4003a70 0x7f15b4003eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fb0 tx=0x7f15b4005830 comp rx=0 tx=0).stop 2024-09-10T22:35:13.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f8ff2b10640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec162110 0x7f8fec1f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f98e4f52640 1 Processor -- start 2024-09-10T22:35:13.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3066398941 shutdown_connections 2024-09-10T22:35:13.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f15d1f6b640 1 --2- 172.21.15.66:0/3066398941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15b4003a70 0x7f15b4003eb0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f15bc006fb0 tx=0x7f15b4005830 comp rx=0 tx=0).stop 2024-09-10T22:35:13.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3066398941 >> 172.21.15.66:0/3066398941 conn(0x7f15cc0b1660 msgr2=0x7f15cc0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f98e4f52640 1 -- start start 2024-09-10T22:35:13.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f8ff2b10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fec0adf50 0x7f8fec142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.400+0000 7f8febfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec162110 0x7f8fec1f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f8febfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec162110 0x7f8fec1f96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35564/0 (socket says 172.21.15.66:35564) 2024-09-10T22:35:13.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f8febfff640 1 -- 172.21.15.66:0/3157755644 learned_addr learned my addr 172.21.15.66:0/3157755644 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.399+0000 7ff9711de640 1 -- 172.21.15.66:0/1734566863 wait complete. 2024-09-10T22:35:13.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7ff9711de640 1 Processor -- start 2024-09-10T22:35:13.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f8ff2b10640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec162880 con 0x7f8fec162110 2024-09-10T22:35:13.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f8ff2b10640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec162700 con 0x7f8fec0adf50 2024-09-10T22:35:13.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3066398941 shutdown_connections 2024-09-10T22:35:13.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f8ff2b10640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec162a00 con 0x7f8fec074430 2024-09-10T22:35:13.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 -- 172.21.15.66:0/4234157929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a74154790 msgr2=0x7f0a7415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 --2- 172.21.15.66:0/4234157929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a74154790 0x7f0a7415e5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0a740b2e10 tx=0x7f0a6802a600 comp rx=0 tx=0).stop 2024-09-10T22:35:13.412 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f8ff10ad640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fec0adf50 0x7f8fec142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 -- 172.21.15.66:0/4234157929 shutdown_connections 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 --2- 172.21.15.66:0/4234157929 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a74069fc0 0x7f0a74070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 --2- 172.21.15.66:0/4234157929 >> v2:172.21.15.66:3300/0 conn(0x7f0a7415a5f0 0x7f0a7406fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 --2- 172.21.15.66:0/4234157929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a74154790 0x7f0a7415e5a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 -- 172.21.15.66:0/4234157929 >> 172.21.15.66:0/4234157929 conn(0x7f0a740b1dd0 msgr2=0x7f0a740b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa77bcbd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7740cf580 0x7fa7740c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa77bcbd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7740d4ad0 0x7fa7740b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa77bcbd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa774161b10 0x7fa7740b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa77bcbd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa774162810 con 0x7fa7740cf580 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa77bcbd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7740ceb60 con 0x7fa7740d4ad0 2024-09-10T22:35:13.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa77bcbd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa774162990 con 0x7fa774161b10 2024-09-10T22:35:13.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa779a59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7740cf580 0x7fa7740c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7ff9711de640 1 -- start start 2024-09-10T22:35:13.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa779a59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7740cf580 0x7fa7740c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35586/0 (socket says 172.21.15.66:35586) 2024-09-10T22:35:13.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa779a59640 1 -- 172.21.15.66:0/1312360375 learned_addr learned my addr 172.21.15.66:0/1312360375 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa779258640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7740d4ad0 0x7fa7740b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f10f228c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ec0ccc30 0x7f10ec0cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7fa77a25a640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa774161b10 0x7fa7740b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 -- 172.21.15.66:0/4234157929 shutdown_connections 2024-09-10T22:35:13.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f8ff08ac640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fec074430 0x7f8fec144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f0a7979b640 1 -- 172.21.15.66:0/4234157929 wait complete. 2024-09-10T22:35:13.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10f228c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ec0cf650 0x7f10ec1592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f50b93c6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b4073bb0 0x7f50b4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ec0ccc30 0x7f10ec0cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10ebfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ec0ccc30 0x7f10ec0cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38142/0 (socket says 172.21.15.66:38142) 2024-09-10T22:35:13.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10ebfff640 1 -- 172.21.15.66:0/970093035 learned_addr learned my addr 172.21.15.66:0/970093035 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50b93c6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b4073bb0 0x7f50b4156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35578/0 (socket says 172.21.15.66:35578) 2024-09-10T22:35:13.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10f228c640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0b0530 0x7f10ec0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10eb7fe640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ec0cf650 0x7f10ec1592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8ff10ad640 1 -- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fec074430 msgr2=0x7f8fec144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8ff10ad640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8fec074430 0x7f8fec144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8ff10ad640 1 -- 172.21.15.66:0/3157755644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec162110 msgr2=0x7f8fec1f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8ff10ad640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8fec162110 0x7f8fec1f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8ff10ad640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec144df0 con 0x7f8fec0adf50 2024-09-10T22:35:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50b93c6640 1 -- 172.21.15.66:0/257072222 learned_addr learned my addr 172.21.15.66:0/257072222 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10f228c640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec070780 con 0x7f10ec0b0530 2024-09-10T22:35:13.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f98e4f52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e006eaf0 0x7f98e009bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50bb62a640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b40b4150 0x7f50b41f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7fa779a59640 1 -- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa774161b10 msgr2=0x7fa7740b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7fa779a59640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa774161b10 0x7fa7740b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7fa779a59640 1 -- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7740d4ad0 msgr2=0x7fa7740b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7fa779a59640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7740d4ad0 0x7fa7740b80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f98e4f52640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98e00d1ab0 0x7f98e009f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50bb62a640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 0x7f50b41550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.422 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8ff10ad640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fec0adf50 0x7f8fec142710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400be20 tx=0x7f8fe400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f98de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e006eaf0 0x7f98e009bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50b415f220 con 0x7f50b4073bb0 2024-09-10T22:35:13.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50b415f0a0 con 0x7f50b4151f80 2024-09-10T22:35:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7fa779a59640 1 -- 172.21.15.66:0/1312360375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa774161370 con 0x7fa7740cf580 2024-09-10T22:35:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7fa77a25a640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa774161b10 0x7fa7740b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7fa779258640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7740d4ad0 0x7fa7740b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f98de59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e006eaf0 0x7f98e009bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38154/0 (socket says 172.21.15.66:38154) 2024-09-10T22:35:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f98de59c640 1 -- 172.21.15.66:0/369654613 learned_addr learned my addr 172.21.15.66:0/369654613 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f50b415f3a0 con 0x7f50b40b4150 2024-09-10T22:35:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50b9bc7640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 0x7f50b41550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f98ddd9b640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98e00d1ab0 0x7f98e009f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f98e4f52640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015df20 0x7f98e01f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f50b8bc5640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b40b4150 0x7f50b41f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4019070 con 0x7f8fec0adf50 2024-09-10T22:35:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe4014440 con 0x7f8fec0adf50 2024-09-10T22:35:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40148d0 con 0x7f8fec0adf50 2024-09-10T22:35:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.402+0000 7f10f0829640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0b0530 0x7f10ec0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98e00d0650 con 0x7f98e015df20 2024-09-10T22:35:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f10f228c640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec070600 con 0x7f10ec0ccc30 2024-09-10T22:35:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f10f228c640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f10ec163ef0 con 0x7f10ec0cf650 2024-09-10T22:35:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98e00d04d0 con 0x7f98e006eaf0 2024-09-10T22:35:13.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f98e00d07d0 con 0x7f98e00d1ab0 2024-09-10T22:35:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f8ff2b10640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec1450d0 con 0x7f8fec0adf50 2024-09-10T22:35:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.401+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3066398941 wait complete. 2024-09-10T22:35:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f15d1f6b640 1 Processor -- start 2024-09-10T22:35:13.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f8ff2b10640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec145690 con 0x7f8fec0adf50 2024-09-10T22:35:13.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f98ded9d640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015df20 0x7f98e01f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9711de640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c072440 0x7ff96c148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9711de640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff96c0cf0c0 0x7ff96c156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9711de640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96c13bf40 0x7ff96c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9711de640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c06ebb0 con 0x7ff96c13bf40 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9711de640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c06ed30 con 0x7ff96c072440 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9711de640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c06eeb0 con 0x7ff96c0cf0c0 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9709dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96c13bf40 0x7ff96c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9709dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96c13bf40 0x7ff96c155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35616/0 (socket says 172.21.15.66:35616) 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff9709dd640 1 -- 172.21.15.66:0/1878473559 learned_addr learned my addr 172.21.15.66:0/1878473559 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff96b7fe640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff96c0cf0c0 0x7ff96c156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f50b9bc7640 1 -- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b40b4150 msgr2=0x7f50b41f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f50b9bc7640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b40b4150 0x7f50b41f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f50b9bc7640 1 -- 172.21.15.66:0/257072222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b4073bb0 msgr2=0x7f50b4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f50b9bc7640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b4073bb0 0x7f50b4156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f50b9bc7640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41555e0 con 0x7f50b4151f80 2024-09-10T22:35:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f50b9bc7640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 0x7f50b41550a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b3e0 tx=0x7f50a400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f0a7979b640 1 Processor -- start 2024-09-10T22:35:13.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7ff96bfff640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c072440 0x7ff96c148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f10ebfff640 1 -- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ec0cf650 msgr2=0x7f10ec1592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f10ebfff640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f10ec0cf650 0x7f10ec1592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f10ebfff640 1 -- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0b0530 msgr2=0x7f10ec0b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f10ebfff640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0b0530 0x7f10ec0b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f10ebfff640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec158820 con 0x7f10ec0ccc30 2024-09-10T22:35:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f0a7979b640 1 -- start start 2024-09-10T22:35:13.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f10f0829640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10ec0b0530 0x7f10ec0b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:13.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7fa779a59640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7740cf580 0x7fa7740c76d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa76401c420 tx=0x7fa76401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7ff96bfff640 1 -- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff96c0cf0c0 msgr2=0x7ff96c156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7ff96bfff640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff96c0cf0c0 0x7ff96c156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7ff96bfff640 1 -- 172.21.15.66:0/1878473559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96c13bf40 msgr2=0x7ff96c155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7ff96bfff640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96c13bf40 0x7ff96c155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f98ddd9b640 1 -- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e006eaf0 msgr2=0x7f98e009bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f98ddd9b640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98e006eaf0 0x7f98e009bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f98ddd9b640 1 -- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015df20 msgr2=0x7f98e01f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f98ddd9b640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98e015df20 0x7f98e01f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f98ddd9b640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e01f7f30 con 0x7f98e00d1ab0 2024-09-10T22:35:13.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a401aa80 con 0x7f50b4151f80 2024-09-10T22:35:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b4156980 con 0x7f50b4151f80 2024-09-10T22:35:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41f0710 con 0x7f50b4151f80 2024-09-10T22:35:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7ff96bfff640 1 -- 172.21.15.66:0/1878473559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c144bd0 con 0x7ff96c072440 2024-09-10T22:35:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.403+0000 7f15d1f6b640 1 -- start start 2024-09-10T22:35:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f15d1f6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15cc204540 0x7f15cc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f15d1f6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f15cc0b55a0 0x7f15cc15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f15d1f6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15cc15d080 0x7f15cc204d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f15d1f6b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc0aefb0 con 0x7f15cc15d080 2024-09-10T22:35:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f15d1f6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc0aee30 con 0x7f15cc204540 2024-09-10T22:35:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f15d1f6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f15cc0af130 con 0x7f15cc0b55a0 2024-09-10T22:35:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f8ff2b10640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8fec073ef0 con 0x7f8fec0adf50 2024-09-10T22:35:13.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.404+0000 7f98ddd9b640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98e00d1ab0 0x7f98e009f510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d000c940 tx=0x7f98d000ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a401f070 con 0x7f50b4151f80 2024-09-10T22:35:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4011630 con 0x7f50b4151f80 2024-09-10T22:35:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764022640 con 0x7fa7740cf580 2024-09-10T22:35:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa764027070 con 0x7fa7740cf580 2024-09-10T22:35:13.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15caffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f15cc0b55a0 0x7f15cc15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764022b70 con 0x7fa7740cf580 2024-09-10T22:35:13.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15caffd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f15cc0b55a0 0x7f15cc15cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59706/0 (socket says 172.21.15.66:59706) 2024-09-10T22:35:13.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15caffd640 1 -- 172.21.15.66:0/3132607100 learned_addr learned my addr 172.21.15.66:0/3132607100 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f0a7979b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a74069fc0 0x7f0a74153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15cb7fe640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15cc204540 0x7f15cc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15cbfff640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15cc15d080 0x7f15cc204d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77414e1e0 con 0x7fa7740cf580 2024-09-10T22:35:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7740c9240 con 0x7fa7740cf580 2024-09-10T22:35:13.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f509f7fe640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f508c001030 con 0x7f50b4151f80 2024-09-10T22:35:13.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15caffd640 1 -- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15cc204540 msgr2=0x7f15cc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15caffd640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15cc204540 0x7f15cc0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f15caffd640 1 -- 172.21.15.66:0/3132607100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15cc15d080 msgr2=0x7f15cc204d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f15caffd640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15cc15d080 0x7f15cc204d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f15caffd640 1 -- 172.21.15.66:0/3132607100 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc0b3ef0 con 0x7f15cc0b55a0 2024-09-10T22:35:13.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f10ebfff640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ec0ccc30 0x7f10ec0cdf00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f10dc02bd60 tx=0x7f10dc0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc003b80 con 0x7f10ec0ccc30 2024-09-10T22:35:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.405+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10dc045070 con 0x7f10ec0ccc30 2024-09-10T22:35:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc034480 con 0x7f10ec0ccc30 2024-09-10T22:35:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f15cbfff640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15cc15d080 0x7f15cc204d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:13.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7ff96bfff640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c072440 0x7ff96c148850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff964009000 tx=0x7ff964003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f10f228c640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec0c6f30 con 0x7f10ec0ccc30 2024-09-10T22:35:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a7979b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a7415a5f0 0x7f0a7414a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a7979b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a7414e650 0x7f0a74156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a7979b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7406e8d0 con 0x7f0a74069fc0 2024-09-10T22:35:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a7979b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a74070b80 con 0x7f0a7415a5f0 2024-09-10T22:35:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a7979b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7406ea50 con 0x7f0a7414e650 2024-09-10T22:35:13.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a72ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a74069fc0 0x7f0a74153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f10f228c640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec158ac0 con 0x7f10ec0ccc30 2024-09-10T22:35:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10c4001030 con 0x7f10ec0ccc30 2024-09-10T22:35:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7741618c0 con 0x7fa7740cf580 2024-09-10T22:35:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0007bb0 con 0x7f98e00d1ab0 2024-09-10T22:35:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d0007e40 con 0x7f98e00d1ab0 2024-09-10T22:35:13.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d000f6b0 con 0x7f98e00d1ab0 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e01f8210 con 0x7f98e00d1ab0 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e00caf40 con 0x7f98e00d1ab0 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a72ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a74069fc0 0x7f0a74153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35640/0 (socket says 172.21.15.66:35640) 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a72ffd640 1 -- 172.21.15.66:0/3983670385 learned_addr learned my addr 172.21.15.66:0/3983670385 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a72ffd640 1 -- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a7414e650 msgr2=0x7f0a74156140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a72ffd640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a7414e650 0x7f0a74156140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f0a72ffd640 1 -- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a7415a5f0 msgr2=0x7f0a7414a860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f0a72ffd640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a7415a5f0 0x7f0a7414a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f0a72ffd640 1 -- 172.21.15.66:0/3983670385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a740af150 con 0x7f0a74069fc0 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f0a72ffd640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a74069fc0 0x7f0a74153a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a68000c00 tx=0x7f0a68003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6803f070 con 0x7f0a74069fc0 2024-09-10T22:35:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f15caffd640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f15cc0b55a0 0x7f15cc15cb40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15c000d710 tx=0x7f15c000dbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f50a40117d0 con 0x7f50b4151f80 2024-09-10T22:35:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964004040 con 0x7ff96c072440 2024-09-10T22:35:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c144eb0 con 0x7ff96c072440 2024-09-10T22:35:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c145470 con 0x7ff96c072440 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9640041e0 con 0x7ff96c072440 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.406+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8fe401d3b0 con 0x7f8fec0adf50 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f8fe97fa640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8fd8078040 0x7f8fd807a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe409a050 con 0x7f8fec0adf50 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f8ff08ac640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8fd8078040 0x7f8fd807a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f8fe97fa640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f8fd8080260 0x7f8fd8082680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fe401dbb0 con 0x7f8fd8080260 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8fe40a1050 con 0x7f8fec0adf50 2024-09-10T22:35:13.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f8febfff640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f8fd8080260 0x7f8fd8082680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0014070 con 0x7f15cc0b55a0 2024-09-10T22:35:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98e009fa50 con 0x7f98e00d1ab0 2024-09-10T22:35:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a68004cf0 con 0x7f0a74069fc0 2024-09-10T22:35:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.407+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6803f070 con 0x7f0a74069fc0 2024-09-10T22:35:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a740af3f0 con 0x7f0a74069fc0 2024-09-10T22:35:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f8ff08ac640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8fd8078040 0x7f8fd807a500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8fec15f040 tx=0x7f8fe0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc0b4190 con 0x7f15cc0b55a0 2024-09-10T22:35:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc0b4750 con 0x7f15cc0b55a0 2024-09-10T22:35:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15c000de20 con 0x7f15cc0b55a0 2024-09-10T22:35:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c000ba90 con 0x7f15cc0b55a0 2024-09-10T22:35:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964004610 con 0x7ff96c072440 2024-09-10T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a741507e0 con 0x7f0a74069fc0 2024-09-10T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f50a9ffb640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5098077ff0 0x7f509807a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f50b93c6640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5098077ff0 0x7f509807a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.408+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f50a40a2090 con 0x7f50b4151f80 2024-09-10T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.409+0000 7f50b93c6640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5098077ff0 0x7f509807a4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009920 tx=0x7f50ac004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.409+0000 7f50a9ffb640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7f5098080190 0x7f50980825b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.409+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5098082c60 con 0x7f5098080190 2024-09-10T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.409+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f50a40a9050 con 0x7f50b4151f80 2024-09-10T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.409+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f15cc0cd230 con 0x7f15cc0b55a0 2024-09-10T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.409+0000 7f50b8bc5640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7f5098080190 0x7f50980825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.409+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa764021070 con 0x7fa7740cf580 2024-09-10T22:35:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f10dc034620 con 0x7f10ec0ccc30 2024-09-10T22:35:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a54001030 con 0x7f0a74069fc0 2024-09-10T22:35:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7fa76a7fc640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa758078070 0x7fa75807a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7f50b8bc5640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7f5098080190 0x7f50980825b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa7640aafb0 con 0x7fa7740cf580 2024-09-10T22:35:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff948001030 con 0x7ff96c072440 2024-09-10T22:35:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7fa779258640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa758078070 0x7fa75807a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7fa76a7fc640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fa758080290 0x7fa7580826b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 --> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa758082d60 con 0x7fa758080290 2024-09-10T22:35:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa7640ab3a0 con 0x7fa7740cf580 2024-09-10T22:35:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f15c000bc30 con 0x7f15cc0b55a0 2024-09-10T22:35:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff96402e050 con 0x7ff96c072440 2024-09-10T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.410+0000 7f10e8ff9640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f10d8078060 0x7f10d807a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f10dc0bc970 con 0x7f10ec0ccc30 2024-09-10T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f10e8ff9640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f10d8080280 0x7f10d80826a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10dc039ea0 con 0x7f10d8080280 2024-09-10T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f10dc0bcd60 con 0x7f10ec0ccc30 2024-09-10T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f98d0020030 con 0x7f98e00d1ab0 2024-09-10T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7fa77a25a640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fa758080290 0x7fa7580826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== osd.5 v2:172.21.15.94:6802/2846740531 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5098082c60 con 0x7f5098080190 2024-09-10T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f8febfff640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f8fd8080260 0x7f8fd8082680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f10eb7fe640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f10d8078060 0x7f10d807a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7fa779258640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa758078070 0x7fa75807a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa770033000 tx=0x7fa770004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f10f0829640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f10d8080280 0x7f10d80826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f15b3fff640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f15b8077f70 0x7f15b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f15c009a950 con 0x7f15cc0b55a0 2024-09-10T22:35:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f98ceffd640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f98c0078070 0x7f98c007a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f15b3fff640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7f15b80801f0 0x7f15b8082610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 --> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15c001ce70 con 0x7f15b80801f0 2024-09-10T22:35:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f15c009ad40 con 0x7f15cc0b55a0 2024-09-10T22:35:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0a68004ee0 con 0x7f0a74069fc0 2024-09-10T22:35:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f98d009a0e0 con 0x7f98e00d1ab0 2024-09-10T22:35:13.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== osd.7 v2:172.21.15.94:6810/3602692462 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fe401dbb0 con 0x7f8fd8080260 2024-09-10T22:35:13.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7ff968ff9640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff954078070 0x7ff95407a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f98ceffd640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7f98c0080270 0x7f98c0082690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff9640ba860 con 0x7ff96c072440 2024-09-10T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7ff96b7fe640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff954078070 0x7ff95407a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7ff968ff9640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7ff954080290 0x7ff9540826b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 --> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff964039e90 con 0x7ff954080290 2024-09-10T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff9640bf2a0 con 0x7ff96c072440 2024-09-10T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f0a5b7fe640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a60078370 0x7f0a6007a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f15cb7fe640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f15b8077f70 0x7f15b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f10f0829640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f10d8080280 0x7f10d80826a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.411+0000 7fa77a25a640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fa758080290 0x7fa7580826b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f10eb7fe640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f10d8078060 0x7f10d807a520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10ec072480 tx=0x7f10e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7ff96b7fe640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff954078070 0x7ff95407a530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff96c156af0 tx=0x7ff958009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f15cbfff640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7f15b80801f0 0x7f15b8082610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a68004a00 con 0x7f0a74069fc0 2024-09-10T22:35:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98c0082d40 con 0x7f98c0080270 2024-09-10T22:35:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f98d00a1050 con 0x7f98e00d1ab0 2024-09-10T22:35:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== osd.1 v2:172.21.15.66:6800/1998688775 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa758082d60 con 0x7fa758080290 2024-09-10T22:35:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f0a5b7fe640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7f0a60081090 0x7f0a60083530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 --> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a60083c00 con 0x7f0a60081090 2024-09-10T22:35:13.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0a6807f320 con 0x7f0a74069fc0 2024-09-10T22:35:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.412+0000 7f98de59c640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f98c0078070 0x7f98c007a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f98ded9d640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7f98c0080270 0x7f98c0082690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f15cb7fe640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f15b8077f70 0x7f15b807a430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f15bc033000 tx=0x7f15bc0019d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.413+0000 7f0a737fe640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7f0a60081090 0x7f0a60083530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7f15cbfff640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7f15b80801f0 0x7f15b8082610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7f0a737fe640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7f0a60081090 0x7f0a60083530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== osd.0 v2:172.21.15.66:6808/1933221850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10dc039ea0 con 0x7f10d8080280 2024-09-10T22:35:13.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7f98de59c640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f98c0078070 0x7f98c007a530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009b00 tx=0x7f98d4005f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7f0a727fc640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a60078370 0x7f0a6007a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7ff9709dd640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7ff954080290 0x7ff9540826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7f0a727fc640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a60078370 0x7f0a6007a830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a74158040 tx=0x7f0a64009220 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7f98ded9d640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7f98c0080270 0x7f98c0082690 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.414+0000 7ff9709dd640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7ff954080290 0x7ff9540826b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.415+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== osd.3 v2:172.21.15.66:6824/2075348943 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a60083c00 con 0x7f0a60081090 2024-09-10T22:35:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.415+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== osd.4 v2:172.21.15.94:6818/3674153549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15c001ce70 con 0x7f15b80801f0 2024-09-10T22:35:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.415+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== osd.6 v2:172.21.15.94:6826/2189915660 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff964039e90 con 0x7ff954080290 2024-09-10T22:35:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.415+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== osd.2 v2:172.21.15.66:6816/3996225872 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98c0082d40 con 0x7f98c0080270 2024-09-10T22:35:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.438+0000 7f509f7fe640 1 -- 172.21.15.66:0/257072222 --> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f508c002dd0 con 0x7f5098080190 2024-09-10T22:35:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.439+0000 7f50a9ffb640 1 -- 172.21.15.66:0/257072222 <== osd.5 v2:172.21.15.94:6802/2846740531 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f508c002dd0 con 0x7f5098080190 2024-09-10T22:35:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.439+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7f5098080190 msgr2=0x7f50980825b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.439+0000 7f50bb62a640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7f5098080190 0x7f50980825b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.440+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5098077ff0 msgr2=0x7f509807a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.440+0000 7f50bb62a640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5098077ff0 0x7f509807a4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009920 tx=0x7f50ac004ab0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.440+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 msgr2=0x7f50b41550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.440+0000 7f50bb62a640 1 --2- 172.21.15.66:0/257072222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b4151f80 0x7f50b41550a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b3e0 tx=0x7f50a400b8a0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.440+0000 7f50b9bc7640 1 -- 172.21.15.66:0/257072222 reap_dead start 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.440+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 shutdown_connections 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.440+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 >> 172.21.15.66:0/257072222 conn(0x7f50b40b27c0 msgr2=0x7f50b40b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.441+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 shutdown_connections 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.441+0000 7f50bb62a640 1 -- 172.21.15.66:0/257072222 wait complete. 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.443+0000 7f8ff2b10640 1 -- 172.21.15.66:0/3157755644 --> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8fec15f110 con 0x7f8fd8080260 2024-09-10T22:35:13.471 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.443+0000 7f8fe97fa640 1 -- 172.21.15.66:0/3157755644 <== osd.7 v2:172.21.15.94:6810/3602692462 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8fec15f110 con 0x7f8fd8080260 2024-09-10T22:35:13.472 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 --> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7741618c0 con 0x7fa758080290 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f8fd8080260 msgr2=0x7f8fd8082680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7f8fd2ffd640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f8fd8080260 0x7f8fd8082680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8fd8078040 msgr2=0x7f8fd807a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7f8fd2ffd640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8fd8078040 0x7f8fd807a500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8fec15f040 tx=0x7f8fe0009040 comp rx=0 tx=0).stop 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fec0adf50 msgr2=0x7f8fec142710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7f8fd2ffd640 1 --2- 172.21.15.66:0/3157755644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fec0adf50 0x7f8fec142710 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400be20 tx=0x7f8fe400bf10 comp rx=0 tx=0).stop 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7f8ff10ad640 1 -- 172.21.15.66:0/3157755644 reap_dead start 2024-09-10T22:35:13.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.444+0000 7fa76a7fc640 1 -- 172.21.15.66:0/1312360375 <== osd.1 v2:172.21.15.66:6800/1998688775 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa76c001930 con 0x7fa758080290 2024-09-10T22:35:13.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3157755644 shutdown_connections 2024-09-10T22:35:13.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3157755644 >> 172.21.15.66:0/3157755644 conn(0x7f8fec06cd00 msgr2=0x7f8fec0b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3157755644 shutdown_connections 2024-09-10T22:35:13.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7f8fd2ffd640 1 -- 172.21.15.66:0/3157755644 wait complete. 2024-09-10T22:35:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fa758080290 msgr2=0x7fa7580826b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7fa77bcbd640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fa758080290 0x7fa7580826b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 --> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f10c4002dd0 con 0x7f10d8080280 2024-09-10T22:35:13.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7f10e8ff9640 1 -- 172.21.15.66:0/970093035 <== osd.0 v2:172.21.15.66:6808/1933221850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f10c4002dd0 con 0x7f10d8080280 2024-09-10T22:35:13.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 --> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f15cc0cd230 con 0x7f15b80801f0 2024-09-10T22:35:13.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa758078070 msgr2=0x7fa75807a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7fa77bcbd640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa758078070 0x7fa75807a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa770033000 tx=0x7fa770004730 comp rx=0 tx=0).stop 2024-09-10T22:35:13.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7740cf580 msgr2=0x7fa7740c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.445+0000 7fa77bcbd640 1 --2- 172.21.15.66:0/1312360375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa7740cf580 0x7fa7740c76d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa76401c420 tx=0x7fa76401c8e0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7fa77a25a640 1 -- 172.21.15.66:0/1312360375 reap_dead start 2024-09-10T22:35:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 shutdown_connections 2024-09-10T22:35:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 >> 172.21.15.66:0/1312360375 conn(0x7fa7740d3fb0 msgr2=0x7fa7740b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 shutdown_connections 2024-09-10T22:35:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7fa77bcbd640 1 -- 172.21.15.66:0/1312360375 wait complete. 2024-09-10T22:35:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 --> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff948002dd0 con 0x7ff954080290 2024-09-10T22:35:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.446+0000 7f15b3fff640 1 -- 172.21.15.66:0/3132607100 <== osd.4 v2:172.21.15.94:6818/3674153549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f15cc0cd230 con 0x7f15b80801f0 2024-09-10T22:35:13.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7ff968ff9640 1 -- 172.21.15.66:0/1878473559 <== osd.6 v2:172.21.15.94:6826/2189915660 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff948002dd0 con 0x7ff954080290 2024-09-10T22:35:13.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7f15b80801f0 msgr2=0x7f15b8082610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7f15b80801f0 0x7f15b8082610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f10d8080280 msgr2=0x7f10d80826a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f10d8080280 0x7f10d80826a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f10d8078060 msgr2=0x7f10d807a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f10d8078060 0x7f10d807a520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10ec072480 tx=0x7f10e0009040 comp rx=0 tx=0).stop 2024-09-10T22:35:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ec0ccc30 msgr2=0x7f10ec0cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 --2- 172.21.15.66:0/970093035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10ec0ccc30 0x7f10ec0cdf00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f10dc02bd60 tx=0x7f10dc0045e0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f15b8077f70 msgr2=0x7f15b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f15b8077f70 0x7f15b807a430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f15bc033000 tx=0x7f15bc0019d0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f15cc0b55a0 msgr2=0x7f15cc15cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10f0829640 1 -- 172.21.15.66:0/970093035 reap_dead start 2024-09-10T22:35:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 shutdown_connections 2024-09-10T22:35:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 >> 172.21.15.66:0/970093035 conn(0x7f10ec0b1660 msgr2=0x7f10ec157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 --2- 172.21.15.66:0/3132607100 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f15cc0b55a0 0x7f15cc15cb40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15c000d710 tx=0x7f15c000dbd0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15cbfff640 1 -- 172.21.15.66:0/3132607100 reap_dead start 2024-09-10T22:35:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7ff954080290 msgr2=0x7ff9540826b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 shutdown_connections 2024-09-10T22:35:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f10d27fc640 1 -- 172.21.15.66:0/970093035 wait complete. 2024-09-10T22:35:13.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 --> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a54002dd0 con 0x7f0a60081090 2024-09-10T22:35:13.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 shutdown_connections 2024-09-10T22:35:13.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 >> 172.21.15.66:0/3132607100 conn(0x7f15cc0b1660 msgr2=0x7f15cc157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7ff9711de640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7ff954080290 0x7ff9540826b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 shutdown_connections 2024-09-10T22:35:13.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7f15d1f6b640 1 -- 172.21.15.66:0/3132607100 wait complete. 2024-09-10T22:35:13.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.447+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 --> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98e01f83e0 con 0x7f98c0080270 2024-09-10T22:35:13.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7f0a5b7fe640 1 -- 172.21.15.66:0/3983670385 <== osd.3 v2:172.21.15.66:6824/2075348943 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a54002dd0 con 0x7f0a60081090 2024-09-10T22:35:13.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff954078070 msgr2=0x7ff95407a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7ff9711de640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff954078070 0x7ff95407a530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff96c156af0 tx=0x7ff958009040 comp rx=0 tx=0).stop 2024-09-10T22:35:13.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c072440 msgr2=0x7ff96c148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7ff9711de640 1 --2- 172.21.15.66:0/1878473559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff96c072440 0x7ff96c148850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff964009000 tx=0x7ff964003ba0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7ff9709dd640 1 -- 172.21.15.66:0/1878473559 reap_dead start 2024-09-10T22:35:13.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 shutdown_connections 2024-09-10T22:35:13.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 >> 172.21.15.66:0/1878473559 conn(0x7ff96c0b16d0 msgr2=0x7ff96c0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.448+0000 7f98ceffd640 1 -- 172.21.15.66:0/369654613 <== osd.2 v2:172.21.15.66:6816/3996225872 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f98e01f83e0 con 0x7f98c0080270 2024-09-10T22:35:13.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7f0a60081090 msgr2=0x7f0a60083530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7f0a7979b640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7f0a60081090 0x7f0a60083530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 shutdown_connections 2024-09-10T22:35:13.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7ff9711de640 1 -- 172.21.15.66:0/1878473559 wait complete. 2024-09-10T22:35:13.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7f98c0080270 msgr2=0x7f98c0082690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7f98e4f52640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7f98c0080270 0x7f98c0082690 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f98c0078070 msgr2=0x7f98c007a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7f98e4f52640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f98c0078070 0x7f98c007a530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009b00 tx=0x7f98d4005f30 comp rx=0 tx=0).stop 2024-09-10T22:35:13.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.449+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98e00d1ab0 msgr2=0x7f98e009f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f98e4f52640 1 --2- 172.21.15.66:0/369654613 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98e00d1ab0 0x7f98e009f510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98d000c940 tx=0x7f98d000ce00 comp rx=0 tx=0).stop 2024-09-10T22:35:13.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f98ded9d640 1 -- 172.21.15.66:0/369654613 reap_dead start 2024-09-10T22:35:13.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 shutdown_connections 2024-09-10T22:35:13.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 >> 172.21.15.66:0/369654613 conn(0x7f98e009aa70 msgr2=0x7f98e00d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 shutdown_connections 2024-09-10T22:35:13.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f98e4f52640 1 -- 172.21.15.66:0/369654613 wait complete. 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a60078370 msgr2=0x7f0a6007a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a7979b640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a60078370 0x7f0a6007a830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a74158040 tx=0x7f0a64009220 comp rx=0 tx=0).stop 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a74069fc0 msgr2=0x7f0a74153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a7979b640 1 --2- 172.21.15.66:0/3983670385 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a74069fc0 0x7f0a74153a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a68000c00 tx=0x7f0a68003b20 comp rx=0 tx=0).stop 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a737fe640 1 -- 172.21.15.66:0/3983670385 reap_dead start 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 shutdown_connections 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 >> 172.21.15.66:0/3983670385 conn(0x7f0a740b1dd0 msgr2=0x7f0a7406fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.450+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 shutdown_connections 2024-09-10T22:35:13.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.451+0000 7f0a7979b640 1 -- 172.21.15.66:0/3983670385 wait complete. 2024-09-10T22:35:13.489 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:13.490 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T22:35:13.490 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-09-10T22:35:13.491 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:13.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T22:35:13.491 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T22:35:13.492 INFO:teuthology.orchestra.run.smithi066.stdout:60129542153 2024-09-10T22:35:13.493 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T22:35:13.493 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-09-10T22:35:13.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T22:35:13.494 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:13.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T22:35:13.495 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-09-10T22:35:13.496 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T22:35:13.496 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:13.497 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T22:35:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.712+0000 7f9db1847640 1 -- 172.21.15.66:0/4119319272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d98004f40 msgr2=0x7f9d9800ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.712+0000 7f9db1847640 1 --2- 172.21.15.66:0/4119319272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d98004f40 0x7f9d9800ffd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9da400c640 tx=0x7f9d98005830 comp rx=0 tx=0).stop 2024-09-10T22:35:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.712+0000 7f9db1847640 1 -- 172.21.15.66:0/4119319272 shutdown_connections 2024-09-10T22:35:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.713+0000 7f9db1847640 1 --2- 172.21.15.66:0/4119319272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d98004f40 0x7f9d9800ffd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.713+0000 7f9db1847640 1 -- 172.21.15.66:0/4119319272 >> 172.21.15.66:0/4119319272 conn(0x7f9dac0b1660 msgr2=0x7f9dac0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.713+0000 7f9db1847640 1 -- 172.21.15.66:0/4119319272 shutdown_connections 2024-09-10T22:35:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.714+0000 7f1c04f71640 1 -- 172.21.15.66:0/3067535598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 msgr2=0x7f1be8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.714+0000 7f1c04f71640 1 --2- 172.21.15.66:0/3067535598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 0x7f1be8004c80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8001e60 tx=0x7f1be8005830 comp rx=0 tx=0).stop 2024-09-10T22:35:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.714+0000 7f1c04f71640 1 -- 172.21.15.66:0/3067535598 shutdown_connections 2024-09-10T22:35:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.714+0000 7f1c04f71640 1 --2- 172.21.15.66:0/3067535598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 0x7f1be8004c80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.714+0000 7f1c04f71640 1 -- 172.21.15.66:0/3067535598 >> 172.21.15.66:0/3067535598 conn(0x7f1c000cf9e0 msgr2=0x7f1c000cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.714+0000 7f1c04f71640 1 -- 172.21.15.66:0/3067535598 shutdown_connections 2024-09-10T22:35:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.713+0000 7f9db1847640 1 -- 172.21.15.66:0/4119319272 wait complete. 2024-09-10T22:35:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.715+0000 7f9db1847640 1 Processor -- start 2024-09-10T22:35:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.716+0000 7f9db1847640 1 -- start start 2024-09-10T22:35:13.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.714+0000 7f1c04f71640 1 -- 172.21.15.66:0/3067535598 wait complete. 2024-09-10T22:35:13.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.716+0000 7f1c04f71640 1 Processor -- start 2024-09-10T22:35:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f28ffec6640 1 -- 172.21.15.66:0/156016429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f8071290 msgr2=0x7f28f806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f28ffec6640 1 --2- 172.21.15.66:0/156016429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f8071290 0x7f28f806f6f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009880 tx=0x7f28e802a390 comp rx=0 tx=0).stop 2024-09-10T22:35:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f28ffec6640 1 -- 172.21.15.66:0/156016429 shutdown_connections 2024-09-10T22:35:13.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f28ffec6640 1 --2- 172.21.15.66:0/156016429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f8071290 0x7f28f806f6f0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009880 tx=0x7f28e802a390 comp rx=0 tx=0).stop 2024-09-10T22:35:13.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f28ffec6640 1 -- 172.21.15.66:0/156016429 >> 172.21.15.66:0/156016429 conn(0x7f28f80b28c0 msgr2=0x7f28f80b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9db1847640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d98004f40 0x7f9dac204c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f28ffec6640 1 -- 172.21.15.66:0/156016429 shutdown_connections 2024-09-10T22:35:13.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9db1847640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dac0c94a0 0x7f9dac203a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9db1847640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dac0c3ff0 0x7f9dac0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9db1847640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac070780 con 0x7f9d98004f40 2024-09-10T22:35:13.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9db1847640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac070600 con 0x7f9dac0c3ff0 2024-09-10T22:35:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f28ffec6640 1 -- 172.21.15.66:0/156016429 wait complete. 2024-09-10T22:35:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d98004f40 0x7f9dac204c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9db1847640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9dac0cc520 con 0x7f9dac0c94a0 2024-09-10T22:35:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d98004f40 0x7f9dac204c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35708/0 (socket says 172.21.15.66:35708) 2024-09-10T22:35:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f9daaffd640 1 -- 172.21.15.66:0/2207608728 learned_addr learned my addr 172.21.15.66:0/2207608728 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.717+0000 7f1c04f71640 1 -- start start 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f1c04f71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 0x7f1c000abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f9da27fc640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dac0c94a0 0x7f9dac203a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f9daaffd640 1 -- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dac0c94a0 msgr2=0x7f9dac203a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f9daaffd640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dac0c94a0 0x7f9dac203a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f9daaffd640 1 -- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dac0c3ff0 msgr2=0x7f9dac0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f9daaffd640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dac0c3ff0 0x7f9dac0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f9daaffd640 1 -- 172.21.15.66:0/2207608728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac0c6c80 con 0x7f9d98004f40 2024-09-10T22:35:13.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f1c04f71640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c00156c70 0x7f1c000ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f1c04f71640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c000cc970 0x7f1c00151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f9da27fc640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dac0c94a0 0x7f9dac203a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:13.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f1bfeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c00156c70 0x7f1c000ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f1bfeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c00156c70 0x7f1c000ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38254/0 (socket says 172.21.15.66:38254) 2024-09-10T22:35:13.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f1bfeffd640 1 -- 172.21.15.66:0/1202301747 learned_addr learned my addr 172.21.15.66:0/1202301747 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.718+0000 7f1bff7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 0x7f1c000abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f1c04f71640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000736c0 con 0x7f1be8004800 2024-09-10T22:35:13.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f9daaffd640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d98004f40 0x7f9dac204c00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b420 tx=0x7f9d9c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f1c04f71640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000adb40 con 0x7f1c00156c70 2024-09-10T22:35:13.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f1c04f71640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1c00073840 con 0x7f1c000cc970 2024-09-10T22:35:13.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f9da8ff9640 1 -- 172.21.15.66:0/2207608728 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c011830 con 0x7f9d98004f40 2024-09-10T22:35:13.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f9da8ff9640 1 -- 172.21.15.66:0/2207608728 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d9c016070 con 0x7f9d98004f40 2024-09-10T22:35:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f9da8ff9640 1 -- 172.21.15.66:0/2207608728 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c011d60 con 0x7f9d98004f40 2024-09-10T22:35:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f28ffec6640 1 Processor -- start 2024-09-10T22:35:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f28ffec6640 1 -- start start 2024-09-10T22:35:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f8ab4832640 1 -- 172.21.15.66:0/173004902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aa8005ab0 msgr2=0x7f8aa8036520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f8ab4832640 1 --2- 172.21.15.66:0/173004902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aa8005ab0 0x7f8aa8036520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009830 tx=0x7f8aa402a4d0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1836866113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406fc70 msgr2=0x7f0ed4073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 --2- 172.21.15.66:0/1836866113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406fc70 0x7f0ed4073460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4070360 tx=0x7f0ec402c3b0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0f928a9640 1 -- 172.21.15.66:0/3771381917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84054ee0 msgr2=0x7f0f8404fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0f928a9640 1 --2- 172.21.15.66:0/3771381917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84054ee0 0x7f0f8404fe40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c06c900 tx=0x7f0f8c0a5cb0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f0f928a9640 1 -- 172.21.15.66:0/3771381917 shutdown_connections 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f0f928a9640 1 --2- 172.21.15.66:0/3771381917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f84054ee0 0x7f0f8404fe40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f0f928a9640 1 --2- 172.21.15.66:0/3771381917 >> v2:172.21.15.66:3300/0 conn(0x7f0f84037990 0x7f0f84008a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f0f928a9640 1 -- 172.21.15.66:0/3771381917 >> 172.21.15.66:0/3771381917 conn(0x7f0f8401e8e0 msgr2=0x7f0f84055640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.719+0000 7f1bfffff640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c000cc970 0x7f1c00151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f1bff7fe640 1 -- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c000cc970 msgr2=0x7f1c00151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f1bff7fe640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c000cc970 0x7f1c00151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f1bff7fe640 1 -- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c00156c70 msgr2=0x7f1c000ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f1bff7fe640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c00156c70 0x7f1c000ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f1bff7fe640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c000cdba0 con 0x7f1be8004800 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f1bff7fe640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 0x7f1c000abb40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c730 tx=0x7f1bf000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1836866113 shutdown_connections 2024-09-10T22:35:13.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 --2- 172.21.15.66:0/1836866113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406fc70 0x7f0ed4073460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 --2- 172.21.15.66:0/1836866113 >> v2:172.21.15.94:3300/0 conn(0x7f0ed415e6e0 0x7f0ed40cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1836866113 >> 172.21.15.66:0/1836866113 conn(0x7f0ed40b1660 msgr2=0x7f0ed40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1836866113 shutdown_connections 2024-09-10T22:35:13.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1836866113 wait complete. 2024-09-10T22:35:13.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f8ab4832640 1 -- 172.21.15.66:0/173004902 shutdown_connections 2024-09-10T22:35:13.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f8ab4832640 1 --2- 172.21.15.66:0/173004902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aa8005ab0 0x7f8aa8036520 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f8ab4832640 1 --2- 172.21.15.66:0/173004902 >> v2:172.21.15.66:3300/0 conn(0x7f8aa800ba00 0x7f8aa8035fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f8ab4832640 1 -- 172.21.15.66:0/173004902 >> 172.21.15.66:0/173004902 conn(0x7f8aa8054560 msgr2=0x7f8aa8054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f9db1847640 1 -- 172.21.15.66:0/2207608728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac205140 con 0x7f9d98004f40 2024-09-10T22:35:13.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.720+0000 7f9db1847640 1 -- 172.21.15.66:0/2207608728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac0c7200 con 0x7f9d98004f40 2024-09-10T22:35:13.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28ffec6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f806cb20 0x7f28f81f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28ffec6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28f80af1d0 0x7f28f8142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28ffec6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28f81f4b20 0x7f28f81433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28ffec6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f28f806d420 con 0x7f28f806cb20 2024-09-10T22:35:13.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28ffec6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28f806d2a0 con 0x7f28f80af1d0 2024-09-10T22:35:13.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28ffec6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f28f80d15d0 con 0x7f28f81f4b20 2024-09-10T22:35:13.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f332d401640 1 -- 172.21.15.66:0/804862699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3328161740 msgr2=0x7f33280b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f332d401640 1 --2- 172.21.15.66:0/804862699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3328161740 0x7f33280b4b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f33280b55c0 tx=0x7f332002a540 comp rx=0 tx=0).stop 2024-09-10T22:35:13.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28fdc62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f806cb20 0x7f28f81f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f332d401640 1 -- 172.21.15.66:0/804862699 shutdown_connections 2024-09-10T22:35:13.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f332d401640 1 --2- 172.21.15.66:0/804862699 >> v2:172.21.15.94:3300/0 conn(0x7f33280b45f0 0x7f33280ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f332d401640 1 --2- 172.21.15.66:0/804862699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3328161740 0x7f33280b4b90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f332d401640 1 -- 172.21.15.66:0/804862699 >> 172.21.15.66:0/804862699 conn(0x7f332809d110 msgr2=0x7f33280d3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28fdc62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f806cb20 0x7f28f81f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35742/0 (socket says 172.21.15.66:35742) 2024-09-10T22:35:13.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28fdc62640 1 -- 172.21.15.66:0/696293482 learned_addr learned my addr 172.21.15.66:0/696293482 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f332d401640 1 -- 172.21.15.66:0/804862699 shutdown_connections 2024-09-10T22:35:13.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.721+0000 7f28fe463640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28f81f4b20 0x7f28f81433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f9db1847640 1 -- 172.21.15.66:0/2207608728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac0ce9c0 con 0x7f9d98004f40 2024-09-10T22:35:13.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28fdc62640 1 -- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28f81f4b20 msgr2=0x7f28f81433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28fdc62640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28f81f4b20 0x7f28f81433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28fdc62640 1 -- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28f80af1d0 msgr2=0x7f28f8142eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:35:13.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28fdc62640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28f80af1d0 0x7f28f8142eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28fdc62640 1 -- 172.21.15.66:0/696293482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f81592c0 con 0x7f28f806cb20 2024-09-10T22:35:13.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f1be7fff640 1 -- 172.21.15.66:0/1202301747 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0017070 con 0x7f1be8004800 2024-09-10T22:35:13.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28fdc62640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f806cb20 0x7f28f81f45e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28f400c480 tx=0x7f28f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f332d401640 1 -- 172.21.15.66:0/804862699 wait complete. 2024-09-10T22:35:13.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f0eda5cd640 1 Processor -- start 2024-09-10T22:35:13.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f1be7fff640 1 -- 172.21.15.66:0/1202301747 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bf0005380 con 0x7f1be8004800 2024-09-10T22:35:13.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f1be7fff640 1 -- 172.21.15.66:0/1202301747 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf000a8a0 con 0x7f1be8004800 2024-09-10T22:35:13.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f1c04f71640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c000cde80 con 0x7f1be8004800 2024-09-10T22:35:13.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f1c04f71640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0014fdc0 con 0x7f1be8004800 2024-09-10T22:35:13.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f0eda5cd640 1 -- start start 2024-09-10T22:35:13.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f8ab4832640 1 -- 172.21.15.66:0/173004902 shutdown_connections 2024-09-10T22:35:13.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28ee7fc640 1 -- 172.21.15.66:0/696293482 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4005480 con 0x7f28f806cb20 2024-09-10T22:35:13.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f8ab4832640 1 -- 172.21.15.66:0/173004902 wait complete. 2024-09-10T22:35:13.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f0f928a9640 1 -- 172.21.15.66:0/3771381917 shutdown_connections 2024-09-10T22:35:13.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28ee7fc640 1 -- 172.21.15.66:0/696293482 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28f40035f0 con 0x7f28f806cb20 2024-09-10T22:35:13.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.722+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f81595a0 con 0x7f28f806cb20 2024-09-10T22:35:13.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f81442f0 con 0x7f28f806cb20 2024-09-10T22:35:13.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f332d401640 1 Processor -- start 2024-09-10T22:35:13.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.723+0000 7f0eda5cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406f890 0x7f0ed4201e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.723+0000 7f0eda5cd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ed406fc70 0x7f0ed40c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0ed3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406f890 0x7f0ed4201e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0ed3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406f890 0x7f0ed4201e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35756/0 (socket says 172.21.15.66:35756) 2024-09-10T22:35:13.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0eda5cd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed415e6e0 0x7f0ed4200870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f332d401640 1 -- start start 2024-09-10T22:35:13.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f28ee7fc640 1 -- 172.21.15.66:0/696293482 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4003aa0 con 0x7f28f806cb20 2024-09-10T22:35:13.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.723+0000 7f0f928a9640 1 -- 172.21.15.66:0/3771381917 wait complete. 2024-09-10T22:35:13.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0ed37fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ed406fc70 0x7f0ed40c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0ed3fff640 1 -- 172.21.15.66:0/1784225554 learned_addr learned my addr 172.21.15.66:0/1784225554 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0ed8b6a640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed415e6e0 0x7f0ed4200870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4073ec0 con 0x7f0ed406f890 2024-09-10T22:35:13.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4073d40 con 0x7f0ed415e6e0 2024-09-10T22:35:13.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed406e510 con 0x7f0ed406fc70 2024-09-10T22:35:13.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.723+0000 7f9da8ff9640 1 -- 172.21.15.66:0/2207608728 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9d9c01b460 con 0x7f9d98004f40 2024-09-10T22:35:13.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f9da8ff9640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d8c078040 0x7f9d8c07a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f9da8ff9640 1 -- 172.21.15.66:0/2207608728 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9d9c0999b0 con 0x7f9d98004f40 2024-09-10T22:35:13.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.724+0000 7f0ed37fe640 1 -- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed415e6e0 msgr2=0x7f0ed4200870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed37fe640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed415e6e0 0x7f0ed4200870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed37fe640 1 -- 172.21.15.66:0/1784225554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406f890 msgr2=0x7f0ed4201e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed37fe640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406f890 0x7f0ed4201e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed37fe640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed415ba60 con 0x7f0ed406fc70 2024-09-10T22:35:13.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed37fe640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ed406fc70 0x7f0ed40c7690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c8f0 tx=0x7f0ec800cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f9da27fc640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d8c078040 0x7f9d8c07a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed0ff9640 1 -- 172.21.15.66:0/1784225554 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8007bb0 con 0x7f0ed406fc70 2024-09-10T22:35:13.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f332d401640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280b45f0 0x7f3328204af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed415bd40 con 0x7f0ed406fc70 2024-09-10T22:35:13.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed415c300 con 0x7f0ed406fc70 2024-09-10T22:35:13.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0f928a9640 1 Processor -- start 2024-09-10T22:35:13.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0f928a9640 1 -- start start 2024-09-10T22:35:13.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed0ff9640 1 -- 172.21.15.66:0/1784225554 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ec800ae40 con 0x7f0ed406fc70 2024-09-10T22:35:13.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.725+0000 7f0ed0ff9640 1 -- 172.21.15.66:0/1784225554 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8005280 con 0x7f0ed406fc70 2024-09-10T22:35:13.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.726+0000 7f1be7fff640 1 -- 172.21.15.66:0/1202301747 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1bf000aa40 con 0x7f1be8004800 2024-09-10T22:35:13.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f1be7fff640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1bec078070 0x7f1bec07a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f1c04f71640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be0005290 con 0x7f1be8004800 2024-09-10T22:35:13.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f1bfeffd640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1bec078070 0x7f1bec07a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f1be7fff640 1 -- 172.21.15.66:0/1202301747 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1bf0099390 con 0x7f1be8004800 2024-09-10T22:35:13.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 -- 172.21.15.66:0/989743794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 msgr2=0x7f383406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f28ee7fc640 1 -- 172.21.15.66:0/696293482 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f28f4003c40 con 0x7f28f806cb20 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f928a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84037990 0x7f0f8417d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f928a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f8403a390 0x7f0f840e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f928a9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f84054ee0 0x7f0f840078a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f928a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84050710 con 0x7f0f84037990 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f928a9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84050590 con 0x7f0f8403a390 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f928a9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0f84050890 con 0x7f0f84054ee0 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84037990 0x7f0f8417d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84037990 0x7f0f8417d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35766/0 (socket says 172.21.15.66:35766) 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 -- 172.21.15.66:0/57300208 learned_addr learned my addr 172.21.15.66:0/57300208 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f90e46640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f84054ee0 0x7f0f840078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 -- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f84054ee0 msgr2=0x7f0f840078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f84054ee0 0x7f0f840078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 -- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f8403a390 msgr2=0x7f0f840e4820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f8403a390 0x7f0f840e4820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f8bfff640 1 -- 172.21.15.66:0/57300208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8404cb90 con 0x7f0f84037990 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f0f90e46640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f84054ee0 0x7f0f840078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f0f8bfff640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84037990 0x7f0f8417d0f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000e9f0 tx=0x7f0f8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.726+0000 7f8ab4832640 1 Processor -- start 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.726+0000 7f8ab4832640 1 -- start start 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8ab4832640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aa8005ab0 0x7f8aa80ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8ab4832640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aa800ba00 0x7f8aa80360e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8ab4832640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aa801c620 0x7f8aa8048a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8ab4832640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8009ed0 con 0x7f8aa800ba00 2024-09-10T22:35:13.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8ab4832640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8009d50 con 0x7f8aa8005ab0 2024-09-10T22:35:13.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8ab4832640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa800a050 con 0x7f8aa801c620 2024-09-10T22:35:13.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 --2- 172.21.15.66:0/989743794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f383406e6c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f38340b5c50 tx=0x7f382c02b490 comp rx=0 tx=0).stop 2024-09-10T22:35:13.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.727+0000 7f332d401640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33280d0620 0x7f3328200f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f332d401640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3328161740 0x7f33280c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f332d401640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33280cf280 con 0x7f33280d0620 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f332d401640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33280cf100 con 0x7f33280b45f0 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f332d401640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f33280cf400 con 0x7f3328161740 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33280d0620 0x7f3328200f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33280d0620 0x7f3328200f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35768/0 (socket says 172.21.15.66:35768) 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 -- 172.21.15.66:0/675727035 learned_addr learned my addr 172.21.15.66:0/675727035 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aa801c620 0x7f8aa8048a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aa801c620 0x7f8aa8048a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59794/0 (socket says 172.21.15.66:59794) 2024-09-10T22:35:13.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 -- 172.21.15.66:0/989743794 shutdown_connections 2024-09-10T22:35:13.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 -- 172.21.15.66:0/2589477672 learned_addr learned my addr 172.21.15.66:0/2589477672 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 --2- 172.21.15.66:0/989743794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f383406e6c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 --2- 172.21.15.66:0/989743794 >> v2:172.21.15.66:3300/0 conn(0x7f38340cf170 0x7f38340aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 -- 172.21.15.66:0/989743794 >> 172.21.15.66:0/989743794 conn(0x7f38340b1660 msgr2=0x7f38340b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:13.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 -- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3328161740 msgr2=0x7f33280c8e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:35:13.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3328161740 0x7f33280c8e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 -- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280b45f0 msgr2=0x7f3328204af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:35:13.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280b45f0 0x7f3328204af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 -- 172.21.15.66:0/675727035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3328201630 con 0x7f33280d0620 2024-09-10T22:35:13.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f33267fc640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33280d0620 0x7f3328200f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f331000c700 tx=0x7f331000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 -- 172.21.15.66:0/989743794 shutdown_connections 2024-09-10T22:35:13.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f383b072640 1 -- 172.21.15.66:0/989743794 wait complete. 2024-09-10T22:35:13.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 -- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aa8005ab0 msgr2=0x7f8aa80ddf40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:35:13.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aa8005ab0 0x7f8aa80ddf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 -- 172.21.15.66:0/2589477672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aa800ba00 msgr2=0x7f8aa80360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aae59c640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aa800ba00 0x7f8aa80360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aa800ba00 0x7f8aa80360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f1bfeffd640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1bec078070 0x7f1bec07a530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8001e60 tx=0x7f1bf8001610 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aaf59e640 1 -- 172.21.15.66:0/2589477672 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa801e1e0 con 0x7f8aa801c620 2024-09-10T22:35:13.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f8aae59c640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aa800ba00 0x7f8aa80360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:13.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.728+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f806a240 con 0x7f28f806cb20 2024-09-10T22:35:13.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f9da27fc640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d8c078040 0x7f9d8c07a500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0ae750 tx=0x7f9d9400c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f8aaf59e640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aa801c620 0x7f8aa8048a10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c390 tx=0x7f8aa000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb4005290 con 0x7f0ed406fc70 2024-09-10T22:35:13.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f8a977fe640 1 -- 172.21.15.66:0/2589477672 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0017070 con 0x7f8aa801c620 2024-09-10T22:35:13.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa80e22c0 con 0x7f8aa801c620 2024-09-10T22:35:13.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa80e1d00 con 0x7f8aa801c620 2024-09-10T22:35:13.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f8a977fe640 1 -- 172.21.15.66:0/2589477672 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aa00035f0 con 0x7f8aa801c620 2024-09-10T22:35:13.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f8a977fe640 1 -- 172.21.15.66:0/2589477672 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0003a40 con 0x7f8aa801c620 2024-09-10T22:35:13.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.729+0000 7f28ee7fc640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f28dc078160 0x7f28dc07a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.730+0000 7f28fd461640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f28dc078160 0x7f28dc07a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.730+0000 7f0ed0ff9640 1 -- 172.21.15.66:0/1784225554 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0ec8005420 con 0x7f0ed406fc70 2024-09-10T22:35:13.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.730+0000 7f28fd461640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f28dc078160 0x7f28dc07a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28e8034000 tx=0x7f28e8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.730+0000 7f0ed0ff9640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ec0078040 0x7f0ec007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.731+0000 7f383b072640 1 Processor -- start 2024-09-10T22:35:13.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.731+0000 7f0ed0ff9640 1 -- 172.21.15.66:0/1784225554 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ec8099fc0 con 0x7f0ed406fc70 2024-09-10T22:35:13.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.731+0000 7f0ed3fff640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ec0078040 0x7f0ec007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.731+0000 7f383b072640 1 -- start start 2024-09-10T22:35:13.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.731+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a90005290 con 0x7f8aa801c620 2024-09-10T22:35:13.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.731+0000 7f0ed3fff640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ec0078040 0x7f0ec007a500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0133c0 tx=0x7f0ecc004260 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.731+0000 7f8a977fe640 1 -- 172.21.15.66:0/2589477672 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8aa0003be0 con 0x7f8aa801c620 2024-09-10T22:35:13.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.732+0000 7f8a977fe640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a9c077f70 0x7f8a9c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.732+0000 7f8a977fe640 1 -- 172.21.15.66:0/2589477672 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8aa00992b0 con 0x7f8aa801c620 2024-09-10T22:35:13.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.732+0000 7f8aaed9d640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a9c077f70 0x7f8a9c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.763 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.732+0000 7f8aaed9d640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a9c077f70 0x7f8a9c07a430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ab007af60 tx=0x7f8ab006c810 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f383b072640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f3834153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f383b072640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340cd6e0 0x7f38340c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f3838e0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f3834153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f3838e0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f3834153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38296/0 (socket says 172.21.15.66:38296) 2024-09-10T22:35:13.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f3838e0e640 1 -- 172.21.15.66:0/1144684831 learned_addr learned my addr 172.21.15.66:0/1144684831 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:13.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f3833fff640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340cd6e0 0x7f38340c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f331f7fe640 1 -- 172.21.15.66:0/675727035 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310017070 con 0x7f33280d0620 2024-09-10T22:35:13.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f28ee7fc640 1 -- 172.21.15.66:0/696293482 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f28f405dfc0 con 0x7f28f806cb20 2024-09-10T22:35:13.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f383b072640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38340cf170 0x7f38340c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33280c9340 con 0x7f33280d0620 2024-09-10T22:35:13.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38340b22d0 con 0x7f38340cd6e0 2024-09-10T22:35:13.766 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33282053b0 con 0x7f33280d0620 2024-09-10T22:35:13.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f383960f640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38340cf170 0x7f38340c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f0f88ff9640 1 -- 172.21.15.66:0/57300208 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8000cd20 con 0x7f0f84037990 2024-09-10T22:35:13.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f840e4d60 con 0x7f0f84037990 2024-09-10T22:35:13.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f9da8ff9640 1 -- 172.21.15.66:0/2207608728 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9d9c062520 con 0x7f9d98004f40 2024-09-10T22:35:13.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38340b2150 con 0x7f38340ccc30 2024-09-10T22:35:13.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.733+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f84179550 con 0x7f0f84037990 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.734+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f383406ec00 con 0x7f38340cf170 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f383960f640 1 -- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 msgr2=0x7f3834153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f383960f640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f3834153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f383960f640 1 -- 172.21.15.66:0/1144684831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340cd6e0 msgr2=0x7f38340c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f383960f640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340cd6e0 0x7f38340c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f383960f640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38340c98e0 con 0x7f38340cf170 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f3833fff640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340cd6e0 0x7f38340c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f3838e0e640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f3834153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f383960f640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38340cf170 0x7f38340c6280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f382c031000 tx=0x7f382c00a910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33280cf9b0 con 0x7f33280d0620 2024-09-10T22:35:13.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f0ed0ff9640 1 -- 172.21.15.66:0/1784225554 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0ec8014030 con 0x7f0ed406fc70 2024-09-10T22:35:13.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.735+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84008a90 con 0x7f0f84037990 2024-09-10T22:35:13.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.739+0000 7f1be7fff640 1 -- 172.21.15.66:0/1202301747 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1bf0061820 con 0x7f1be8004800 2024-09-10T22:35:13.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.739+0000 7f38317fa640 1 -- 172.21.15.66:0/1144684831 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c037040 con 0x7f38340cf170 2024-09-10T22:35:13.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.739+0000 7f38317fa640 1 -- 172.21.15.66:0/1144684831 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f382c00aa80 con 0x7f38340cf170 2024-09-10T22:35:13.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.739+0000 7f38317fa640 1 -- 172.21.15.66:0/1144684831 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c003e60 con 0x7f38340cf170 2024-09-10T22:35:13.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.740+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38340c3770 con 0x7f38340cf170 2024-09-10T22:35:13.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.740+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f38340c3d30 con 0x7f38340cf170 2024-09-10T22:35:13.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.742+0000 7f8a977fe640 1 -- 172.21.15.66:0/2589477672 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8aa0061e80 con 0x7f8aa801c620 2024-09-10T22:35:13.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.742+0000 7f38317fa640 1 -- 172.21.15.66:0/1144684831 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f382c049020 con 0x7f38340cf170 2024-09-10T22:35:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.743+0000 7f38317fa640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f382007c380 0x7f382007e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.743+0000 7f38317fa640 1 -- 172.21.15.66:0/1144684831 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f382c0b9a60 con 0x7f38340cf170 2024-09-10T22:35:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.743+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38340cc6c0 con 0x7f38340cf170 2024-09-10T22:35:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.743+0000 7f3838e0e640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f382007c380 0x7f382007e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.744+0000 7f3838e0e640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f382007c380 0x7f382007e840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3828005e30 tx=0x7f3828005dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.744+0000 7f0f88ff9640 1 -- 172.21.15.66:0/57300208 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f80004020 con 0x7f0f84037990 2024-09-10T22:35:13.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.745+0000 7f331f7fe640 1 -- 172.21.15.66:0/675727035 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3310005360 con 0x7f33280d0620 2024-09-10T22:35:13.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.745+0000 7f331f7fe640 1 -- 172.21.15.66:0/675727035 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331000a880 con 0x7f33280d0620 2024-09-10T22:35:13.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.745+0000 7f28ee7fc640 1 -- 172.21.15.66:0/696293482 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f28f4062450 con 0x7f28f806cb20 2024-09-10T22:35:13.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.745+0000 7f0f88ff9640 1 -- 172.21.15.66:0/57300208 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80026060 con 0x7f0f84037990 2024-09-10T22:35:13.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.748+0000 7f331f7fe640 1 -- 172.21.15.66:0/675727035 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f331000aa20 con 0x7f33280d0620 2024-09-10T22:35:13.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.749+0000 7f331f7fe640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3308077f70 0x7f330807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.749+0000 7f3326ffd640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3308077f70 0x7f330807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.750+0000 7f331f7fe640 1 -- 172.21.15.66:0/675727035 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f33100999e0 con 0x7f33280d0620 2024-09-10T22:35:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.748+0000 7f0f88ff9640 1 -- 172.21.15.66:0/57300208 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0f800035f0 con 0x7f0f84037990 2024-09-10T22:35:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.749+0000 7f0f88ff9640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0f70077fa0 0x7f0f7007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.749+0000 7f0f88ff9640 1 -- 172.21.15.66:0/57300208 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0f8009aea0 con 0x7f0f84037990 2024-09-10T22:35:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.750+0000 7f0f8b7fe640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0f70077fa0 0x7f0f7007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:13.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.750+0000 7f3326ffd640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3308077f70 0x7f330807a430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3320033000 tx=0x7f3320004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.750+0000 7f38317fa640 1 -- 172.21.15.66:0/1144684831 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f382c0825d0 con 0x7f38340cf170 2024-09-10T22:35:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.750+0000 7f0f8b7fe640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0f70077fa0 0x7f0f7007a460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f8404f0d0 tx=0x7f0f780078a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:13.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.753+0000 7f0f88ff9640 1 -- 172.21.15.66:0/57300208 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0f840e4d60 con 0x7f0f84037990 2024-09-10T22:35:13.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:13.761+0000 7f331f7fe640 1 -- 172.21.15.66:0/675727035 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3310021020 con 0x7f33280d0620 2024-09-10T22:35:14.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.164+0000 7f0eda5cd640 1 -- 172.21.15.66:0/1784225554 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0eb4005580 con 0x7f0ed406fc70 2024-09-10T22:35:14.166 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.165+0000 7f0ed0ff9640 1 -- 172.21.15.66:0/1784225554 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ec806b250 con 0x7f0ed406fc70 2024-09-10T22:35:14.166 INFO:teuthology.orchestra.run.smithi066.stdout:60129542151 2024-09-10T22:35:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.166+0000 7f9db1847640 1 -- 172.21.15.66:0/2207608728 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9dac069810 con 0x7f9d98004f40 2024-09-10T22:35:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.166+0000 7f9da8ff9640 1 -- 172.21.15.66:0/2207608728 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d9c061e20 con 0x7f9d98004f40 2024-09-10T22:35:14.168 INFO:teuthology.orchestra.run.smithi066.stdout:60129542151 2024-09-10T22:35:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 -- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ec0078040 msgr2=0x7f0ec007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ec0078040 0x7f0ec007a500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0133c0 tx=0x7f0ecc004260 comp rx=0 tx=0).stop 2024-09-10T22:35:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 -- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ed406fc70 msgr2=0x7f0ed40c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ed406fc70 0x7f0ed40c7690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c8f0 tx=0x7f0ec800cdb0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 -- 172.21.15.66:0/1784225554 shutdown_connections 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ec0078040 0x7f0ec007a500 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ed415e6e0 0x7f0ed4200870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ed406fc70 0x7f0ed40c7690 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.169+0000 7f0eba7fc640 1 --2- 172.21.15.66:0/1784225554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ed406f890 0x7f0ed4201e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.170+0000 7f0eba7fc640 1 -- 172.21.15.66:0/1784225554 >> 172.21.15.66:0/1784225554 conn(0x7f0ed40b1660 msgr2=0x7f0ed415de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.170+0000 7f1c04f71640 1 -- 172.21.15.66:0/1202301747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1be0005580 con 0x7f1be8004800 2024-09-10T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.170+0000 7f0eba7fc640 1 -- 172.21.15.66:0/1784225554 shutdown_connections 2024-09-10T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.170+0000 7f0eba7fc640 1 -- 172.21.15.66:0/1784225554 wait complete. 2024-09-10T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.170+0000 7f1be7fff640 1 -- 172.21.15.66:0/1202301747 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1bf0066250 con 0x7f1be8004800 2024-09-10T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 -- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d8c078040 msgr2=0x7f9d8c07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d8c078040 0x7f9d8c07a500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0ae750 tx=0x7f9d9400c040 comp rx=0 tx=0).stop 2024-09-10T22:35:14.173 INFO:teuthology.orchestra.run.smithi066.stdout:64424509446 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 -- 172.21.15.66:0/2207608728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d98004f40 msgr2=0x7f9dac204c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d98004f40 0x7f9dac204c00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b420 tx=0x7f9d9c00b8e0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 -- 172.21.15.66:0/2207608728 shutdown_connections 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d8c078040 0x7f9d8c07a500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dac0c3ff0 0x7f9dac0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.171+0000 7f9da1ffb640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9dac0c94a0 0x7f9dac203a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.172+0000 7f9da1ffb640 1 --2- 172.21.15.66:0/2207608728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d98004f40 0x7f9dac204c00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.172+0000 7f9da1ffb640 1 -- 172.21.15.66:0/2207608728 >> 172.21.15.66:0/2207608728 conn(0x7f9dac0b1660 msgr2=0x7f9dac0d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.172+0000 7f9da1ffb640 1 -- 172.21.15.66:0/2207608728 shutdown_connections 2024-09-10T22:35:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.172+0000 7f9da1ffb640 1 -- 172.21.15.66:0/2207608728 wait complete. 2024-09-10T22:35:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.174+0000 7f1be5ffb640 1 -- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1bec078070 msgr2=0x7f1bec07a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.174+0000 7f1be5ffb640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1bec078070 0x7f1bec07a530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8001e60 tx=0x7f1bf8001610 comp rx=0 tx=0).stop 2024-09-10T22:35:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.174+0000 7f1be5ffb640 1 -- 172.21.15.66:0/1202301747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 msgr2=0x7f1c000abb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.174+0000 7f1be5ffb640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 0x7f1c000abb40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c730 tx=0x7f1bf000cbf0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.175+0000 7f1be5ffb640 1 -- 172.21.15.66:0/1202301747 shutdown_connections 2024-09-10T22:35:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.175+0000 7f1be5ffb640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1bec078070 0x7f1bec07a530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.175+0000 7f1be5ffb640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1c000cc970 0x7f1c00151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.175+0000 7f1be5ffb640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1c00156c70 0x7f1c000ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.175+0000 7f1be5ffb640 1 --2- 172.21.15.66:0/1202301747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1be8004800 0x7f1c000abb40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.175+0000 7f1be5ffb640 1 -- 172.21.15.66:0/1202301747 >> 172.21.15.66:0/1202301747 conn(0x7f1c000cf9e0 msgr2=0x7f1c0015a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.175+0000 7f1be5ffb640 1 -- 172.21.15.66:0/1202301747 shutdown_connections 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.176+0000 7f1be5ffb640 1 -- 172.21.15.66:0/1202301747 wait complete. 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.176+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8a90005580 con 0x7f8aa801c620 2024-09-10T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.177+0000 7f8a977fe640 1 -- 172.21.15.66:0/2589477672 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8aa0061780 con 0x7f8aa801c620 2024-09-10T22:35:14.178 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:14.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.177+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f28f80d1a80 con 0x7f28f806cb20 2024-09-10T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.178+0000 7f28ee7fc640 1 -- 172.21.15.66:0/696293482 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28f4061d50 con 0x7f28f806cb20 2024-09-10T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-09-10T22:35:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.180+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3834153740 con 0x7f38340cf170 2024-09-10T22:35:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.181+0000 7f38317fa640 1 -- 172.21.15.66:0/1144684831 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f382c081ed0 con 0x7f38340cf170 2024-09-10T22:35:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f28dc078160 msgr2=0x7f28dc07a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f28ffec6640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f28dc078160 0x7f28dc07a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28e8034000 tx=0x7f28e8004730 comp rx=0 tx=0).stop 2024-09-10T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f806cb20 msgr2=0x7f28f81f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f28ffec6640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f806cb20 0x7f28f81f45e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28f400c480 tx=0x7f28f400c940 comp rx=0 tx=0).stop 2024-09-10T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a9c077f70 msgr2=0x7f8a9c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f8ab4832640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a9c077f70 0x7f8a9c07a430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ab007af60 tx=0x7f8ab006c810 comp rx=0 tx=0).stop 2024-09-10T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aa801c620 msgr2=0x7f8aa8048a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f8ab4832640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aa801c620 0x7f8aa8048a10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c390 tx=0x7f8aa000c850 comp rx=0 tx=0).stop 2024-09-10T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stdout:64424509446 2024-09-10T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.182+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 shutdown_connections 2024-09-10T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f28ffec6640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f28dc078160 0x7f28dc07a620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f28ffec6640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f28f81f4b20 0x7f28f81433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f28ffec6640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28f80af1d0 0x7f28f8142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f28ffec6640 1 --2- 172.21.15.66:0/696293482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28f806cb20 0x7f28f81f45e0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28f400c480 tx=0x7f28f400c940 comp rx=0 tx=0).stop 2024-09-10T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 >> 172.21.15.66:0/696293482 conn(0x7f28f80b28c0 msgr2=0x7f28f80b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 shutdown_connections 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f8ab4832640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a9c077f70 0x7f8a9c07a430 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f8ab4832640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aa801c620 0x7f8aa8048a10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f8ab4832640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aa800ba00 0x7f8aa80360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f8ab4832640 1 --2- 172.21.15.66:0/2589477672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aa8005ab0 0x7f8aa80ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 >> 172.21.15.66:0/2589477672 conn(0x7f8aa8054560 msgr2=0x7f8aa80e2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 shutdown_connections 2024-09-10T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.183+0000 7f28ffec6640 1 -- 172.21.15.66:0/696293482 wait complete. 2024-09-10T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.184+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 shutdown_connections 2024-09-10T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f8ab4832640 1 -- 172.21.15.66:0/2589477672 wait complete. 2024-09-10T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.184+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f382007c380 msgr2=0x7f382007e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.184+0000 7f383b072640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f382007c380 0x7f382007e840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3828005e30 tx=0x7f3828005dc0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.184+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38340cf170 msgr2=0x7f38340c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.184+0000 7f383b072640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38340cf170 0x7f38340c6280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f382c031000 tx=0x7f382c00a910 comp rx=0 tx=0).stop 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 shutdown_connections 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f382007c380 0x7f382007e840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38340cf170 0x7f38340c6280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38340cd6e0 0x7f38340c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 --2- 172.21.15.66:0/1144684831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38340ccc30 0x7f3834153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 >> 172.21.15.66:0/1144684831 conn(0x7f38340b1660 msgr2=0x7f38340d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 shutdown_connections 2024-09-10T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.185+0000 7f383b072640 1 -- 172.21.15.66:0/1144684831 wait complete. 2024-09-10T22:35:14.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.1 2024-09-10T22:35:14.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.3 2024-09-10T22:35:14.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.191+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0f84050000 con 0x7f0f84037990 2024-09-10T22:35:14.193 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.192+0000 7f0f88ff9640 1 -- 172.21.15.66:0/57300208 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0f8404cb90 con 0x7f0f84037990 2024-09-10T22:35:14.193 INFO:teuthology.orchestra.run.smithi066.stdout:64424509446 2024-09-10T22:35:14.196 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.195+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f33280d01f0 con 0x7f33280d0620 2024-09-10T22:35:14.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f331f7fe640 1 -- 172.21.15.66:0/675727035 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3310062550 con 0x7f33280d0620 2024-09-10T22:35:14.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0f70077fa0 msgr2=0x7f0f7007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0f70077fa0 0x7f0f7007a460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f8404f0d0 tx=0x7f0f780078a0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84037990 msgr2=0x7f0f8417d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.197 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84037990 0x7f0f8417d0f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000e9f0 tx=0x7f0f8000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 shutdown_connections 2024-09-10T22:35:14.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0f70077fa0 0x7f0f7007a460 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0f84054ee0 0x7f0f840078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0f8403a390 0x7f0f840e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 --2- 172.21.15.66:0/57300208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f84037990 0x7f0f8417d0f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.198 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.196+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 >> 172.21.15.66:0/57300208 conn(0x7f0f8401e8e0 msgr2=0x7f0f84055ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.198 INFO:teuthology.orchestra.run.smithi066.stdout:64424509446 2024-09-10T22:35:14.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.197+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 shutdown_connections 2024-09-10T22:35:14.199 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.197+0000 7f0f928a9640 1 -- 172.21.15.66:0/57300208 wait complete. 2024-09-10T22:35:14.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-09-10T22:35:14.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3308077f70 msgr2=0x7f330807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3308077f70 0x7f330807a430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3320033000 tx=0x7f3320004730 comp rx=0 tx=0).stop 2024-09-10T22:35:14.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33280d0620 msgr2=0x7f3328200f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:14.200 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33280d0620 0x7f3328200f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f331000c700 tx=0x7f331000cbc0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 shutdown_connections 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3308077f70 0x7f330807a430 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3328161740 0x7f33280c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33280d0620 0x7f3328200f50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 --2- 172.21.15.66:0/675727035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33280b45f0 0x7f3328204af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 >> 172.21.15.66:0/675727035 conn(0x7f332809d110 msgr2=0x7f33280d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 shutdown_connections 2024-09-10T22:35:14.201 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:14.199+0000 7f332d401640 1 -- 172.21.15.66:0/675727035 wait complete. 2024-09-10T22:35:14.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.0 2024-09-10T22:35:14.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-09-10T22:35:14.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.2 2024-09-10T22:35:14.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-10T22:35:14.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-10T22:35:15.190 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T22:35:15.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T22:35:15.200 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T22:35:15.205 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T22:35:15.205 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T22:35:15.206 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T22:35:15.211 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T22:35:15.215 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T22:35:15.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.366+0000 7f58dac03640 1 -- 172.21.15.66:0/1015411568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 msgr2=0x7f58d40b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.366+0000 7f58dac03640 1 --2- 172.21.15.66:0/1015411568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d40b2960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f58d406fe10 tx=0x7f58c402a640 comp rx=0 tx=0).stop 2024-09-10T22:35:15.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7f58dac03640 1 -- 172.21.15.66:0/1015411568 shutdown_connections 2024-09-10T22:35:15.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7f58dac03640 1 --2- 172.21.15.66:0/1015411568 >> v2:172.21.15.94:3300/0 conn(0x7f58d40ae9e0 0x7f58d40aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.366+0000 7fda6df4b640 1 -- 172.21.15.66:0/2143567773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 msgr2=0x7fda60054ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.368 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.366+0000 7fda6df4b640 1 --2- 172.21.15.66:0/2143567773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60054ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fda6807af60 tx=0x7fda680a5c30 comp rx=0 tx=0).stop 2024-09-10T22:35:15.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7f58dac03640 1 --2- 172.21.15.66:0/1015411568 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d40b2960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7f58dac03640 1 -- 172.21.15.66:0/1015411568 >> 172.21.15.66:0/1015411568 conn(0x7f58d406cd00 msgr2=0x7f58d406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7fda6df4b640 1 -- 172.21.15.66:0/2143567773 shutdown_connections 2024-09-10T22:35:15.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7fda6df4b640 1 --2- 172.21.15.66:0/2143567773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60054ca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7f58dac03640 1 -- 172.21.15.66:0/1015411568 shutdown_connections 2024-09-10T22:35:15.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7fda6df4b640 1 --2- 172.21.15.66:0/2143567773 >> v2:172.21.15.94:3300/0 conn(0x7fda60008be0 0x7fda60054760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7fda6df4b640 1 -- 172.21.15.66:0/2143567773 >> 172.21.15.66:0/2143567773 conn(0x7fda6001e890 msgr2=0x7fda60055640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7f58dac03640 1 -- 172.21.15.66:0/1015411568 wait complete. 2024-09-10T22:35:15.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.369+0000 7f58dac03640 1 Processor -- start 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.369+0000 7f58dac03640 1 -- start start 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58dac03640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58d40ae9e0 0x7f58d415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58dac03640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d4201f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58d3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d4201f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58d3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d4201f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35806/0 (socket says 172.21.15.66:35806) 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58d3fff640 1 -- 172.21.15.66:0/2670607594 learned_addr learned my addr 172.21.15.66:0/2670607594 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58dac03640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d41513d0 0x7f58d4168d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58dac03640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40740a0 con 0x7f58d4150ff0 2024-09-10T22:35:15.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7fda6df4b640 1 -- 172.21.15.66:0/2143567773 shutdown_connections 2024-09-10T22:35:15.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58dac03640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4073f20 con 0x7f58d41513d0 2024-09-10T22:35:15.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7f58dac03640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f58d4074220 con 0x7f58d40ae9e0 2024-09-10T22:35:15.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.367+0000 7fda6df4b640 1 -- 172.21.15.66:0/2143567773 wait complete. 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7f58d3fff640 1 -- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58d40ae9e0 msgr2=0x7f58d415bc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7f58d3fff640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58d40ae9e0 0x7f58d415bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7f58d3fff640 1 -- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d41513d0 msgr2=0x7f58d4168d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7f58d3fff640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d41513d0 0x7f58d4168d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7f58d3fff640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d415d260 con 0x7f58d4150ff0 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7f58d3fff640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d4201f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00ca70 tx=0x7f58cc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.373+0000 7f58d17fa640 1 -- 172.21.15.66:0/2670607594 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc003b70 con 0x7f58d4150ff0 2024-09-10T22:35:15.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.373+0000 7f58dac03640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d4168230 con 0x7f58d4150ff0 2024-09-10T22:35:15.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.369+0000 7fda6df4b640 1 Processor -- start 2024-09-10T22:35:15.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.369+0000 7fda6df4b640 1 -- start start 2024-09-10T22:35:15.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda6df4b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda60008be0 0x7fda6004bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda6df4b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60049350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda677fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda60008be0 0x7fda6004bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda677fe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda60008be0 0x7fda6004bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59818/0 (socket says 172.21.15.66:59818) 2024-09-10T22:35:15.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda66ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60049350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda66ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60049350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35810/0 (socket says 172.21.15.66:35810) 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda6df4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda60034220 0x7fda60172100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda677fe640 1 -- 172.21.15.66:0/3216369759 learned_addr learned my addr 172.21.15.66:0/3216369759 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fda60034750 con 0x7fda60008fc0 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fda60050f20 con 0x7fda60034220 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.371+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fda600348d0 con 0x7fda60008be0 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7fda67fff640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda60034220 0x7fda60172100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7fda677fe640 1 -- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda60034220 msgr2=0x7fda60172100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7fda677fe640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda60034220 0x7fda60172100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7fda677fe640 1 -- 172.21.15.66:0/3216369759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 msgr2=0x7fda60049350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7fda677fe640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60049350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.372+0000 7fda677fe640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda600daf10 con 0x7fda60008be0 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.373+0000 7fda66ffd640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60049350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:15.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.373+0000 7fda677fe640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda60008be0 0x7fda6004bd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00e740 tx=0x7fda5c00ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.375+0000 7fda53fff640 1 -- 172.21.15.66:0/3216369759 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00ccc0 con 0x7fda60008be0 2024-09-10T22:35:15.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.373+0000 7f58dac03640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d416c0c0 con 0x7f58d4150ff0 2024-09-10T22:35:15.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.378+0000 7f58d17fa640 1 -- 172.21.15.66:0/2670607594 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58cc003d10 con 0x7f58d4150ff0 2024-09-10T22:35:15.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.379+0000 7f58d17fa640 1 -- 172.21.15.66:0/2670607594 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0052f0 con 0x7f58d4150ff0 2024-09-10T22:35:15.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.379+0000 7f58d17fa640 1 -- 172.21.15.66:0/2670607594 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f58cc005550 con 0x7f58d4150ff0 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.375+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda600d9e70 con 0x7fda60008be0 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.375+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda600da430 con 0x7fda60008be0 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.377+0000 7fda53fff640 1 -- 172.21.15.66:0/3216369759 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda5c00ee50 con 0x7fda60008be0 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.377+0000 7fda51ffb640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6003a390 con 0x7fda60008be0 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.380+0000 7fda53fff640 1 -- 172.21.15.66:0/3216369759 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c010670 con 0x7fda60008be0 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.380+0000 7fda53fff640 1 -- 172.21.15.66:0/3216369759 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fda5c010890 con 0x7fda60008be0 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.381+0000 7fda53fff640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fda4c078070 0x7fda4c07a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.382+0000 7fda53fff640 1 -- 172.21.15.66:0/3216369759 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fda5c09a750 con 0x7fda60008be0 2024-09-10T22:35:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.380+0000 7f58dac03640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b4005290 con 0x7f58d4150ff0 2024-09-10T22:35:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.384+0000 7fda66ffd640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fda4c078070 0x7fda4c07a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.383+0000 7f58d17fa640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f58c0078070 0x7f58c007a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.383+0000 7f58d17fa640 1 -- 172.21.15.66:0/2670607594 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f58cc099cb0 con 0x7f58d4150ff0 2024-09-10T22:35:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.384+0000 7f58d899f640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f58c0078070 0x7f58c007a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.385+0000 7fda66ffd640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fda4c078070 0x7fda4c07a530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fda6000a940 tx=0x7fda6806b330 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.386 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.385+0000 7f58d899f640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f58c0078070 0x7f58c007a530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58c4032000 tx=0x7f58c4004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.385+0000 7fda53fff640 1 -- 172.21.15.66:0/3216369759 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fda5c062c70 con 0x7fda60008be0 2024-09-10T22:35:15.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.389+0000 7f58d17fa640 1 -- 172.21.15.66:0/2670607594 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f58d40d0330 con 0x7f58d4150ff0 2024-09-10T22:35:15.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 -- 172.21.15.66:0/2852492858 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0e2580 msgr2=0x7f5e7c0e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.425 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 --2- 172.21.15.66:0/2852492858 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0e2580 0x7f5e7c0e2a10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e8406c900 tx=0x7f5e840a7960 comp rx=0 tx=0).stop 2024-09-10T22:35:15.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 -- 172.21.15.66:0/2852492858 shutdown_connections 2024-09-10T22:35:15.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 --2- 172.21.15.66:0/2852492858 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c0e2580 0x7f5e7c0e2a10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 --2- 172.21.15.66:0/2852492858 >> v2:172.21.15.94:3300/0 conn(0x7f5e7c005ab0 0x7f5e7c01cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 --2- 172.21.15.66:0/2852492858 >> v2:172.21.15.66:3300/0 conn(0x7f5e7c00ba00 0x7f5e7c01c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 -- 172.21.15.66:0/2852492858 >> 172.21.15.66:0/2852492858 conn(0x7f5e7c054560 msgr2=0x7f5e7c054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 -- 172.21.15.66:0/2852492858 shutdown_connections 2024-09-10T22:35:15.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 -- 172.21.15.66:0/3457082471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44cc161470 msgr2=0x7f44cc0b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f0231eef640 1 -- 172.21.15.66:0/3485725872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c0d0b10 msgr2=0x7f022c168f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f0231eef640 1 --2- 172.21.15.66:0/3485725872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c0d0b10 0x7f022c168f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0224005fd0 tx=0x7f022402a970 comp rx=0 tx=0).stop 2024-09-10T22:35:15.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 --2- 172.21.15.66:0/3457082471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44cc161470 0x7f44cc0b4aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0c7900 tx=0x7f44c402a9a0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 -- 172.21.15.66:0/3457082471 shutdown_connections 2024-09-10T22:35:15.427 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 --2- 172.21.15.66:0/3457082471 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44cc161850 0x7f44cc0d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 --2- 172.21.15.66:0/3457082471 >> v2:172.21.15.94:3300/0 conn(0x7f44cc0d4940 0x7f44cc0d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 --2- 172.21.15.66:0/3457082471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44cc161470 0x7f44cc0b4aa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 -- 172.21.15.66:0/3457082471 >> 172.21.15.66:0/3457082471 conn(0x7f44cc0d3e20 msgr2=0x7f44cc0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 -- 172.21.15.66:0/3457082471 shutdown_connections 2024-09-10T22:35:15.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f44d4201640 1 -- 172.21.15.66:0/3457082471 wait complete. 2024-09-10T22:35:15.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/4105169463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc053b30 msgr2=0x7fb2dc006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/4105169463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc053b30 0x7fb2dc006ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009830 tx=0x7fb2d402a5b0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f0231eef640 1 -- 172.21.15.66:0/3485725872 shutdown_connections 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f0231eef640 1 --2- 172.21.15.66:0/3485725872 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c0d0b10 0x7f022c168f60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f0231eef640 1 -- 172.21.15.66:0/3485725872 >> 172.21.15.66:0/3485725872 conn(0x7f022c073360 msgr2=0x7f022c074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.424+0000 7f0231eef640 1 -- 172.21.15.66:0/3485725872 shutdown_connections 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.424+0000 7f0231eef640 1 -- 172.21.15.66:0/3485725872 wait complete. 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.424+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/4105169463 shutdown_connections 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.424+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/4105169463 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0073e0 0x7fb2dc0e2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.424+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/4105169463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc053b30 0x7fb2dc006ea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.424+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/4105169463 >> 172.21.15.66:0/4105169463 conn(0x7fb2dc03a440 msgr2=0x7fb2dc03a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.424+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/4105169463 shutdown_connections 2024-09-10T22:35:15.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f452309b640 1 -- 172.21.15.66:0/993004965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c0745e0 msgr2=0x7f451c0d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f452309b640 1 --2- 172.21.15.66:0/993004965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c0745e0 0x7f451c0d0140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f450c009830 tx=0x7f450c02a5b0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f452309b640 1 -- 172.21.15.66:0/993004965 shutdown_connections 2024-09-10T22:35:15.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f452309b640 1 --2- 172.21.15.66:0/993004965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c0745e0 0x7f451c0d0140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f452309b640 1 -- 172.21.15.66:0/993004965 >> 172.21.15.66:0/993004965 conn(0x7f451c0d0c90 msgr2=0x7f451c0d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.423+0000 7f5e89f78640 1 -- 172.21.15.66:0/2852492858 wait complete. 2024-09-10T22:35:15.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f44d4201640 1 Processor -- start 2024-09-10T22:35:15.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f0231eef640 1 Processor -- start 2024-09-10T22:35:15.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f44d4201640 1 -- start start 2024-09-10T22:35:15.432 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/4105169463 wait complete. 2024-09-10T22:35:15.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.426+0000 7f0231eef640 1 -- start start 2024-09-10T22:35:15.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f5e89f78640 1 Processor -- start 2024-09-10T22:35:15.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.425+0000 7f5e89f78640 1 -- start start 2024-09-10T22:35:15.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.426+0000 7f452309b640 1 -- 172.21.15.66:0/993004965 shutdown_connections 2024-09-10T22:35:15.433 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.426+0000 7f452309b640 1 -- 172.21.15.66:0/993004965 wait complete. 2024-09-10T22:35:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.426+0000 7f44d4201640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44cc0d4940 0x7f44cc147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d4201640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44cc161470 0x7f44cc0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d4201640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44cc161850 0x7f44cc145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d4201640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc0b0b50 con 0x7f44cc161850 2024-09-10T22:35:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d4201640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44cc0b09d0 con 0x7f44cc161470 2024-09-10T22:35:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d4201640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f44cc0b0cd0 con 0x7f44cc0d4940 2024-09-10T22:35:15.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f0231eef640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f022c0cf4d0 0x7f022c15d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89f78640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c005ab0 0x7f5e7c021480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89f78640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c00ba00 0x7f5e7c04f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89f78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c0e2580 0x7f5e7c051110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89f78640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c0e32e0 con 0x7f5e7c005ab0 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89f78640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c0e3160 con 0x7f5e7c0e2580 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89f78640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c035fe0 con 0x7f5e7c00ba00 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89777640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c0e2580 0x7f5e7c051110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f0231eef640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f022c0d0b10 0x7f022c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d1f9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44cc0d4940 0x7f44cc147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89777640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c0e2580 0x7f5e7c051110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38354/0 (socket says 172.21.15.66:38354) 2024-09-10T22:35:15.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f5e89777640 1 -- 172.21.15.66:0/3637926066 learned_addr learned my addr 172.21.15.66:0/3637926066 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f022c0cf4d0 0x7f022c15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d179c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44cc161470 0x7f44cc0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d179c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44cc161470 0x7f44cc0c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38350/0 (socket says 172.21.15.66:38350) 2024-09-10T22:35:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f44d179c640 1 -- 172.21.15.66:0/2336334757 learned_addr learned my addr 172.21.15.66:0/2336334757 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.427+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f022c0cf4d0 0x7f022c15d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.66:38370/0 (socket says 172.21.15.66:38370) 2024-09-10T22:35:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f0231eef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c15dfb0 0x7f022c1f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f0231eef640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1694a0 con 0x7f022c15dfb0 2024-09-10T22:35:15.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f0231eef640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f022c0aee10 con 0x7f022c0cf4d0 2024-09-10T22:35:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f0231eef640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f022c169620 con 0x7f022c0d0b10 2024-09-10T22:35:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f022affd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f022c0d0b10 0x7f022c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f022affd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f022c0d0b10 0x7f022c0b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59872/0 (socket says 172.21.15.66:59872) 2024-09-10T22:35:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7fb2e8b9b640 1 Processor -- start 2024-09-10T22:35:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f022affd640 1 -- 172.21.15.66:0/1087567322 learned_addr learned my addr 172.21.15.66:0/1087567322 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.438 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7fb2e8b9b640 1 -- start start 2024-09-10T22:35:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f022bfff640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c15dfb0 0x7f022c1f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f44d179c640 1 -- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44cc0d4940 msgr2=0x7f44cc147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f44d179c640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44cc0d4940 0x7f44cc147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f44d179c640 1 -- 172.21.15.66:0/2336334757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44cc161850 msgr2=0x7f44cc145780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f44d179c640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44cc161850 0x7f44cc145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f44d179c640 1 -- 172.21.15.66:0/2336334757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc0c7d20 con 0x7f44cc161470 2024-09-10T22:35:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e83fff640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c00ba00 0x7f5e7c04f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f022affd640 1 -- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f022c0cf4d0 msgr2=0x7f022c15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f022affd640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f022c0cf4d0 0x7f022c15d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f44d179c640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44cc161470 0x7f44cc0c8a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b4d0 tx=0x7f44bc00b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f022affd640 1 -- 172.21.15.66:0/1087567322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c15dfb0 msgr2=0x7f022c1f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f022affd640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c15dfb0 0x7f022c1f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f022affd640 1 -- 172.21.15.66:0/1087567322 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c1f91a0 con 0x7f022c0d0b10 2024-09-10T22:35:15.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f022bfff640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c15dfb0 0x7f022c1f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:15.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f452309b640 1 Processor -- start 2024-09-10T22:35:15.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f452309b640 1 -- start start 2024-09-10T22:35:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e89777640 1 -- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c00ba00 msgr2=0x7f5e7c04f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e88f76640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c005ab0 0x7f5e7c021480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e89777640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c00ba00 0x7f5e7c04f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e89777640 1 -- 172.21.15.66:0/3637926066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c005ab0 msgr2=0x7f5e7c021480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e89777640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c005ab0 0x7f5e7c021480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e89777640 1 -- 172.21.15.66:0/3637926066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c0e18f0 con 0x7f5e7c0e2580 2024-09-10T22:35:15.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.428+0000 7f5e88f76640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c005ab0 0x7f5e7c021480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f5e89777640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c0e2580 0x7f5e7c051110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e84052150 tx=0x7f5e840b5950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f5e817fa640 1 -- 172.21.15.66:0/3637926066 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840bc070 con 0x7f5e7c0e2580 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f5e817fa640 1 -- 172.21.15.66:0/3637926066 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e840b6420 con 0x7f5e7c0e2580 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f5e817fa640 1 -- 172.21.15.66:0/3637926066 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e840b1040 con 0x7f5e7c0e2580 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f44c27fc640 1 -- 172.21.15.66:0/2336334757 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc014070 con 0x7f44cc161470 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f44c27fc640 1 -- 172.21.15.66:0/2336334757 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44bc010960 con 0x7f44cc161470 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f44c27fc640 1 -- 172.21.15.66:0/2336334757 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc010db0 con 0x7f44cc161470 2024-09-10T22:35:15.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c0e1bd0 con 0x7f5e7c0e2580 2024-09-10T22:35:15.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c0dbe60 con 0x7f5e7c0e2580 2024-09-10T22:35:15.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7fb2e8b9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc0073e0 0x7fb2dc0dba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e8b9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2dc053b30 0x7fb2dc1799d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e8b9b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0e2ab0 0x7fb2dc0d7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f452309b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451c0745e0 0x7f451c0ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f452309b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c151ac0 0x7f451c0c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44cc147fb0 con 0x7f44cc161470 2024-09-10T22:35:15.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.429+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44cc0c8270 con 0x7f44cc161470 2024-09-10T22:35:15.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e8b9b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc0e3550 con 0x7fb2dc0073e0 2024-09-10T22:35:15.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f022affd640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f022c0d0b10 0x7f022c0b0400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f022000b470 tx=0x7f022000b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e2d9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0e2ab0 0x7fb2dc0d7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e2d9d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0e2ab0 0x7fb2dc0d7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59898/0 (socket says 172.21.15.66:59898) 2024-09-10T22:35:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e2d9d640 1 -- 172.21.15.66:0/1429453986 learned_addr learned my addr 172.21.15.66:0/1429453986 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc0e33d0 con 0x7fb2dc053b30 2024-09-10T22:35:15.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f452309b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f451c151ea0 0x7f451c1f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc0e36d0 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7fb2e259c640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc0073e0 0x7fb2dc0dba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f452309b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f451c070740 con 0x7f451c0745e0 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f452309b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0705c0 con 0x7f451c151ac0 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f452309b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f451c0708c0 con 0x7f451c151ea0 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f4520e37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451c0745e0 0x7f451c0ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f4520e37640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451c0745e0 0x7f451c0ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35900/0 (socket says 172.21.15.66:35900) 2024-09-10T22:35:15.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.430+0000 7f4520e37640 1 -- 172.21.15.66:0/4265892000 learned_addr learned my addr 172.21.15.66:0/4265892000 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f0213fff640 1 -- 172.21.15.66:0/1087567322 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220015020 con 0x7f022c0d0b10 2024-09-10T22:35:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f0231eef640 1 -- 172.21.15.66:0/1087567322 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022c0b3e20 con 0x7f022c0d0b10 2024-09-10T22:35:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f0213fff640 1 -- 172.21.15.66:0/1087567322 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0220011980 con 0x7f022c0d0b10 2024-09-10T22:35:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f0231eef640 1 -- 172.21.15.66:0/1087567322 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f022c15d7d0 con 0x7f022c0d0b10 2024-09-10T22:35:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f4521638640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f451c151ea0 0x7f451c1f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f451bfff640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c151ac0 0x7f451c0c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7fb2e1d9b640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2dc053b30 0x7fb2dc1799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7fb2e2d9d640 1 -- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2dc053b30 msgr2=0x7fb2dc1799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7fb2e2d9d640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2dc053b30 0x7fb2dc1799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7fb2e2d9d640 1 -- 172.21.15.66:0/1429453986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc0073e0 msgr2=0x7fb2dc0dba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7fb2e2d9d640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc0073e0 0x7fb2dc0dba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7fb2e2d9d640 1 -- 172.21.15.66:0/1429453986 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc0dc140 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7c0e2a10 con 0x7f5e7c0e2580 2024-09-10T22:35:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f4520e37640 1 -- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f451c151ea0 msgr2=0x7f451c1f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f4520e37640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f451c151ea0 0x7f451c1f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f4520e37640 1 -- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c151ac0 msgr2=0x7f451c0c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f4520e37640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c151ac0 0x7f451c0c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7fb2e2d9d640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0e2ab0 0x7fb2dc0d7b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2e402eb50 tx=0x7fb2e406df00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.450 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44cc0751b0 con 0x7f44cc161470 2024-09-10T22:35:15.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.432+0000 7f4520e37640 1 -- 172.21.15.66:0/4265892000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c1f5130 con 0x7f451c0745e0 2024-09-10T22:35:15.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.432+0000 7f451bfff640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c151ac0 0x7f451c0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:15.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.432+0000 7f4520e37640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451c0745e0 0x7f451c0ca4d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f451400cc50 tx=0x7f451400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.451 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.431+0000 7f0213fff640 1 -- 172.21.15.66:0/1087567322 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220015020 con 0x7f022c0d0b10 2024-09-10T22:35:15.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.432+0000 7f45197fa640 1 -- 172.21.15.66:0/4265892000 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514016020 con 0x7f451c0745e0 2024-09-10T22:35:15.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.432+0000 7f44e6b7d640 1 -- 172.21.15.66:0/598084456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 msgr2=0x7f44e00cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.432+0000 7f44e6b7d640 1 --2- 172.21.15.66:0/598084456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 0x7f44e00cc980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f44e00cc490 tx=0x7f44d002a950 comp rx=0 tx=0).stop 2024-09-10T22:35:15.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f022c0d0400 con 0x7f022c0d0b10 2024-09-10T22:35:15.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f45197fa640 1 -- 172.21.15.66:0/4265892000 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4514012420 con 0x7f451c0745e0 2024-09-10T22:35:15.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c1f5410 con 0x7f451c0745e0 2024-09-10T22:35:15.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f44e6b7d640 1 -- 172.21.15.66:0/598084456 shutdown_connections 2024-09-10T22:35:15.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f44e6b7d640 1 --2- 172.21.15.66:0/598084456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 0x7f44e00cc980 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f45197fa640 1 -- 172.21.15.66:0/4265892000 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451400ce60 con 0x7f451c0745e0 2024-09-10T22:35:15.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f44e6b7d640 1 --2- 172.21.15.66:0/598084456 >> v2:172.21.15.66:3300/0 conn(0x7f44e0070600 0x7f44e0157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f44e6b7d640 1 -- 172.21.15.66:0/598084456 >> 172.21.15.66:0/598084456 conn(0x7f44e00b1660 msgr2=0x7f44e00b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.432+0000 7f44c27fc640 1 -- 172.21.15.66:0/2336334757 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f44bc00f3c0 con 0x7f44cc161470 2024-09-10T22:35:15.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7fb2d2ffd640 1 -- 172.21.15.66:0/1429453986 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e408f070 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7fb2d2ffd640 1 -- 172.21.15.66:0/1429453986 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2e40746e0 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7fb2d2ffd640 1 -- 172.21.15.66:0/1429453986 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4074bf0 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c1f59d0 con 0x7f451c0745e0 2024-09-10T22:35:15.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc0dc420 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc178ec0 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f5e817fa640 1 -- 172.21.15.66:0/3637926066 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5e84064020 con 0x7f5e7c0e2580 2024-09-10T22:35:15.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f5e817fa640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e68078040 0x7f5e6807a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f5e817fa640 1 -- 172.21.15.66:0/3637926066 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5e841360d0 con 0x7f5e7c0e2580 2024-09-10T22:35:15.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f44c27fc640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44b0078040 0x7f44b007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.433+0000 7f44c27fc640 1 -- 172.21.15.66:0/2336334757 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f44bc099df0 con 0x7f44cc161470 2024-09-10T22:35:15.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f44e6b7d640 1 -- 172.21.15.66:0/598084456 shutdown_connections 2024-09-10T22:35:15.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f5e88f76640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e68078040 0x7f5e6807a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f0213fff640 1 -- 172.21.15.66:0/1087567322 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0220011b70 con 0x7f022c0d0b10 2024-09-10T22:35:15.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f44d1f9d640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44b0078040 0x7f44b007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.434+0000 7f44e6b7d640 1 -- 172.21.15.66:0/598084456 wait complete. 2024-09-10T22:35:15.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.435+0000 7f44d1f9d640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44b0078040 0x7f44b007a500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44c4035000 tx=0x7f44c403a040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.435+0000 7f0213fff640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0218078170 0x7f021807a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7f45197fa640 1 -- 172.21.15.66:0/4265892000 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f451401a3b0 con 0x7f451c0745e0 2024-09-10T22:35:15.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.435+0000 7f022b7fe640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0218078170 0x7f021807a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c070d70 con 0x7f451c0745e0 2024-09-10T22:35:15.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7f0213fff640 1 -- 172.21.15.66:0/1087567322 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f02200992f0 con 0x7f022c0d0b10 2024-09-10T22:35:15.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7f5e88f76640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e68078040 0x7f5e6807a500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800a670 tx=0x7f5e780081d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7fb2d2ffd640 1 -- 172.21.15.66:0/1429453986 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb2e4094450 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7f022b7fe640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0218078170 0x7f021807a630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc0570c0 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7f45197fa640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4508077fc0 0x7f450807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.436+0000 7f44e6b7d640 1 Processor -- start 2024-09-10T22:35:15.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.437+0000 7f45197fa640 1 -- 172.21.15.66:0/4265892000 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f451409a0d0 con 0x7f451c0745e0 2024-09-10T22:35:15.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.437+0000 7f44e6b7d640 1 -- start start 2024-09-10T22:35:15.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.437+0000 7f451bfff640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4508077fc0 0x7f450807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.437+0000 7fb2d2ffd640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb2c4078040 0x7fb2c407a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.437+0000 7f5e817fa640 1 -- 172.21.15.66:0/3637926066 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5e840aa070 con 0x7f5e7c0e2580 2024-09-10T22:35:15.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.437+0000 7f451bfff640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4508077fc0 0x7f450807a480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4510007fd0 tx=0x7f4510009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.437+0000 7fb2e259c640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb2c4078040 0x7fb2c407a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7fb2e259c640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb2c4078040 0x7fb2c407a500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8001600 tx=0x7fb2d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7fb2d2ffd640 1 -- 172.21.15.66:0/1429453986 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb2e4115130 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7f44e6b7d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44e0070600 0x7f44e0073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7f44e6b7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 0x7f44e00ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7f44e4919640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44e0070600 0x7f44e0073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7f44e4919640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44e0070600 0x7f44e0073fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:59908/0 (socket says 172.21.15.66:59908) 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7f44e4919640 1 -- 172.21.15.66:0/3225177854 learned_addr learned my addr 172.21.15.66:0/3225177854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.438+0000 7f44e6b7d640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44e00cf650 0x7f44e00ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44dffff640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 0x7f44e00ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44e6b7d640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f44e006e690 con 0x7f44e00cf650 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44e6b7d640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44e006e510 con 0x7f44e00cc520 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44e6b7d640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f44e006e810 con 0x7f44e0070600 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44e511a640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44e00cf650 0x7f44e00ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44e4919640 1 -- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 msgr2=0x7f44e00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44e4919640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 0x7f44e00ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.439+0000 7f44e4919640 1 -- 172.21.15.66:0/3225177854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44e00cf650 msgr2=0x7f44e00ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.440+0000 7f44e4919640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44e00cf650 0x7f44e00ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.440+0000 7f44e4919640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e015eb30 con 0x7f44e0070600 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.440+0000 7f44dffff640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 0x7f44e00ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.440+0000 7f44e511a640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44e00cf650 0x7f44e00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.440+0000 7f44e4919640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44e0070600 0x7f44e0073fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9f0 tx=0x7f44d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.440+0000 7f0213fff640 1 -- 172.21.15.66:0/1087567322 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f022000d040 con 0x7f022c0d0b10 2024-09-10T22:35:15.466 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.442+0000 7f44c27fc640 1 -- 172.21.15.66:0/2336334757 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f44bc062310 con 0x7f44cc161470 2024-09-10T22:35:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.443+0000 7f45197fa640 1 -- 172.21.15.66:0/4265892000 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4514020070 con 0x7f451c0745e0 2024-09-10T22:35:15.467 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.445+0000 7fb2d2ffd640 1 -- 172.21.15.66:0/1429453986 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb2e40dd5c0 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.444+0000 7f44dd7fa640 1 -- 172.21.15.66:0/3225177854 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d800cd80 con 0x7f44e0070600 2024-09-10T22:35:15.468 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.445+0000 7f44dd7fa640 1 -- 172.21.15.66:0/3225177854 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44d8010430 con 0x7f44e0070600 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.445+0000 7f44dd7fa640 1 -- 172.21.15.66:0/3225177854 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8010840 con 0x7f44e0070600 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.446+0000 7f44e6b7d640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e015eda0 con 0x7f44e0070600 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.446+0000 7f44e6b7d640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e015f360 con 0x7f44e0070600 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.449+0000 7f44dd7fa640 1 -- 172.21.15.66:0/3225177854 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f44d80109e0 con 0x7f44e0070600 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.450+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e0157740 con 0x7f44e0070600 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.450+0000 7f44dd7fa640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44c0078040 0x7f44c007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.451+0000 7f44dd7fa640 1 -- 172.21.15.66:0/3225177854 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f44d809a890 con 0x7f44e0070600 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.451+0000 7f44dffff640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44c0078040 0x7f44c007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.452+0000 7f44dffff640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44c0078040 0x7f44c007a500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f44e015d6a0 tx=0x7f44c8007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:15.469 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.456+0000 7f44dd7fa640 1 -- 172.21.15.66:0/3225177854 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f44d8062de0 con 0x7f44e0070600 2024-09-10T22:35:15.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.803+0000 7fda51ffb640 1 -- 172.21.15.66:0/3216369759 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fda60173e50 con 0x7fda60008be0 2024-09-10T22:35:15.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.803+0000 7fda53fff640 1 -- 172.21.15.66:0/3216369759 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fda5c0676a0 con 0x7fda60008be0 2024-09-10T22:35:15.805 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-09-10T22:35:15.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.805+0000 7f58dac03640 1 -- 172.21.15.66:0/2670607594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f58b4005580 con 0x7f58d4150ff0 2024-09-10T22:35:15.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.805+0000 7f58d17fa640 1 -- 172.21.15.66:0/2670607594 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58cc062820 con 0x7f58d4150ff0 2024-09-10T22:35:15.807 INFO:teuthology.orchestra.run.smithi066.stdout:60129542152 2024-09-10T22:35:15.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.807+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fda4c078070 msgr2=0x7fda4c07a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.807+0000 7fda6df4b640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fda4c078070 0x7fda4c07a530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fda6000a940 tx=0x7fda6806b330 comp rx=0 tx=0).stop 2024-09-10T22:35:15.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.807+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda60008be0 msgr2=0x7fda6004bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.807+0000 7fda6df4b640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda60008be0 0x7fda6004bd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00e740 tx=0x7fda5c00ec00 comp rx=0 tx=0).stop 2024-09-10T22:35:15.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.807+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 shutdown_connections 2024-09-10T22:35:15.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.808+0000 7fda6df4b640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fda4c078070 0x7fda4c07a530 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.808+0000 7fda6df4b640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fda60034220 0x7fda60172100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.808+0000 7fda6df4b640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fda60008fc0 0x7fda60049350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.808+0000 7fda6df4b640 1 --2- 172.21.15.66:0/3216369759 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fda60008be0 0x7fda6004bd00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.808+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 >> 172.21.15.66:0/3216369759 conn(0x7fda6001e890 msgr2=0x7fda60172d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.808+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 shutdown_connections 2024-09-10T22:35:15.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.808+0000 7fda6df4b640 1 -- 172.21.15.66:0/3216369759 wait complete. 2024-09-10T22:35:15.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.809+0000 7f58baffd640 1 -- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f58c0078070 msgr2=0x7f58c007a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.809+0000 7f58baffd640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f58c0078070 0x7f58c007a530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58c4032000 tx=0x7f58c4004730 comp rx=0 tx=0).stop 2024-09-10T22:35:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.809+0000 7f58baffd640 1 -- 172.21.15.66:0/2670607594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 msgr2=0x7f58d4201f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.809+0000 7f58baffd640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d4201f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00ca70 tx=0x7f58cc00cf30 comp rx=0 tx=0).stop 2024-09-10T22:35:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.809+0000 7f58baffd640 1 -- 172.21.15.66:0/2670607594 shutdown_connections 2024-09-10T22:35:15.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.809+0000 7f58baffd640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f58c0078070 0x7f58c007a530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.809+0000 7f58baffd640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58d41513d0 0x7f58d4168d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.810+0000 7f58baffd640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58d4150ff0 0x7f58d4201f00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.810+0000 7f58baffd640 1 --2- 172.21.15.66:0/2670607594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f58d40ae9e0 0x7f58d415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.810+0000 7f58baffd640 1 -- 172.21.15.66:0/2670607594 >> 172.21.15.66:0/2670607594 conn(0x7f58d406cd00 msgr2=0x7f58d40afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.810+0000 7f58baffd640 1 -- 172.21.15.66:0/2670607594 shutdown_connections 2024-09-10T22:35:15.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.810+0000 7f58baffd640 1 -- 172.21.15.66:0/2670607594 wait complete. 2024-09-10T22:35:15.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.1 2024-09-10T22:35:15.828 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.3 2024-09-10T22:35:15.829 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.844+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb2dc0dc5f0 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.846 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.845+0000 7fb2d2ffd640 1 -- 172.21.15.66:0/1429453986 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb2e40e6330 con 0x7fb2dc0e2ab0 2024-09-10T22:35:15.846 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb2c4078040 msgr2=0x7fb2c407a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb2c4078040 0x7fb2c407a500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8001600 tx=0x7fb2d8009040 comp rx=0 tx=0).stop 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0e2ab0 msgr2=0x7fb2dc0d7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0e2ab0 0x7fb2dc0d7b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2e402eb50 tx=0x7fb2e406df00 comp rx=0 tx=0).stop 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 shutdown_connections 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb2c4078040 0x7fb2c407a500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb2dc0e2ab0 0x7fb2dc0d7b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb2dc053b30 0x7fb2dc1799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 --2- 172.21.15.66:0/1429453986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2dc0073e0 0x7fb2dc0dba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.847+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 >> 172.21.15.66:0/1429453986 conn(0x7fb2dc03a440 msgr2=0x7fb2dc00dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.849 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.848+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 shutdown_connections 2024-09-10T22:35:15.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.848+0000 7fb2e8b9b640 1 -- 172.21.15.66:0/1429453986 wait complete. 2024-09-10T22:35:15.850 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.849+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f44e00008d0 con 0x7f44e0070600 2024-09-10T22:35:15.851 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.849+0000 7f44dd7fa640 1 -- 172.21.15.66:0/3225177854 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44d8067810 con 0x7f44e0070600 2024-09-10T22:35:15.851 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:15.853 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.851+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44c0078040 msgr2=0x7f44c007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.851+0000 7f44d6ffd640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44c0078040 0x7f44c007a500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f44e015d6a0 tx=0x7f44c8007e90 comp rx=0 tx=0).stop 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.851+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44e0070600 msgr2=0x7f44e0073fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.851+0000 7f44d6ffd640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44e0070600 0x7f44e0073fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9f0 tx=0x7f44d800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 shutdown_connections 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44c0078040 0x7f44c007a500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44e00cf650 0x7f44e00ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44e00cc520 0x7f44e00ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 --2- 172.21.15.66:0/3225177854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44e0070600 0x7f44e0073fc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 >> 172.21.15.66:0/3225177854 conn(0x7f44e00b1660 msgr2=0x7f44e00b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 shutdown_connections 2024-09-10T22:35:15.854 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.852+0000 7f44d6ffd640 1 -- 172.21.15.66:0/3225177854 wait complete. 2024-09-10T22:35:15.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-10T22:35:15.861 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-10T22:35:15.865 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.869+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5e7c0388c0 con 0x7f5e7c0e2580 2024-09-10T22:35:15.870 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.869+0000 7f5e817fa640 1 -- 172.21.15.66:0/3637926066 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5e840febb0 con 0x7f5e7c0e2580 2024-09-10T22:35:15.871 INFO:teuthology.orchestra.run.smithi066.stdout:60129542153 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e68078040 msgr2=0x7f5e6807a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e68078040 0x7f5e6807a500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800a670 tx=0x7f5e780081d0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c0e2580 msgr2=0x7f5e7c051110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c0e2580 0x7f5e7c051110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e84052150 tx=0x7f5e840b5950 comp rx=0 tx=0).stop 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 shutdown_connections 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e68078040 0x7f5e6807a500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c0e2580 0x7f5e7c051110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e7c00ba00 0x7f5e7c04f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 --2- 172.21.15.66:0/3637926066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e7c005ab0 0x7f5e7c021480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.871+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 >> 172.21.15.66:0/3637926066 conn(0x7f5e7c054560 msgr2=0x7f5e7c034c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.872+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 shutdown_connections 2024-09-10T22:35:15.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.872+0000 7f5e89f78640 1 -- 172.21.15.66:0/3637926066 wait complete. 2024-09-10T22:35:15.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.874+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f451c0b6c10 con 0x7f451c0745e0 2024-09-10T22:35:15.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.874+0000 7f45197fa640 1 -- 172.21.15.66:0/4265892000 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4514062cb0 con 0x7f451c0745e0 2024-09-10T22:35:15.875 INFO:teuthology.orchestra.run.smithi066.stdout:64424509448 2024-09-10T22:35:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.874+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f022c0d1240 con 0x7f022c0d0b10 2024-09-10T22:35:15.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.875+0000 7f0213fff640 1 -- 172.21.15.66:0/1087567322 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0220061d70 con 0x7f022c0d0b10 2024-09-10T22:35:15.876 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f44cc0b4aa0 con 0x7f44cc161470 2024-09-10T22:35:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4508077fc0 msgr2=0x7f450807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4508077fc0 0x7f450807a480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4510007fd0 tx=0x7f4510009040 comp rx=0 tx=0).stop 2024-09-10T22:35:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451c0745e0 msgr2=0x7f451c0ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451c0745e0 0x7f451c0ca4d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f451400cc50 tx=0x7f451400a430 comp rx=0 tx=0).stop 2024-09-10T22:35:15.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0218078170 msgr2=0x7f021807a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0218078170 0x7f021807a630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).stop 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f022c0d0b10 msgr2=0x7f022c0b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f022c0d0b10 0x7f022c0b0400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f022000b470 tx=0x7f022000b930 comp rx=0 tx=0).stop 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 shutdown_connections 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4508077fc0 0x7f450807a480 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f451c151ea0 0x7f451c1f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f451c151ac0 0x7f451c0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 --2- 172.21.15.66:0/4265892000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f451c0745e0 0x7f451c0ca4d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 shutdown_connections 2024-09-10T22:35:15.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0218078170 0x7f021807a630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 >> 172.21.15.66:0/4265892000 conn(0x7f451c0d0c90 msgr2=0x7f451c15b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f022c15dfb0 0x7f022c1f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f022c0d0b10 0x7f022c0b0400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 --2- 172.21.15.66:0/1087567322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f022c0cf4d0 0x7f022c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 >> 172.21.15.66:0/1087567322 conn(0x7f022c073360 msgr2=0x7f022c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 shutdown_connections 2024-09-10T22:35:15.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f452309b640 1 -- 172.21.15.66:0/4265892000 wait complete. 2024-09-10T22:35:15.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 shutdown_connections 2024-09-10T22:35:15.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f0211ffb640 1 -- 172.21.15.66:0/1087567322 wait complete. 2024-09-10T22:35:15.882 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.877+0000 7f44c27fc640 1 -- 172.21.15.66:0/2336334757 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f44bc066d40 con 0x7f44cc161470 2024-09-10T22:35:15.882 INFO:teuthology.orchestra.run.smithi066.stdout:64424509447 2024-09-10T22:35:15.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44b0078040 msgr2=0x7f44b007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44b0078040 0x7f44b007a500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44c4035000 tx=0x7f44c403a040 comp rx=0 tx=0).stop 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44cc161470 msgr2=0x7f44cc0c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44cc161470 0x7f44cc0c8a70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b4d0 tx=0x7f44bc00b990 comp rx=0 tx=0).stop 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 shutdown_connections 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f44b0078040 0x7f44b007a500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f44cc161850 0x7f44cc145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44cc161470 0x7f44cc0c8a70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 --2- 172.21.15.66:0/2336334757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f44cc0d4940 0x7f44cc147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 >> 172.21.15.66:0/2336334757 conn(0x7f44cc0d3e20 msgr2=0x7f44cc161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:15.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 shutdown_connections 2024-09-10T22:35:15.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:15.881+0000 7f44d4201640 1 -- 172.21.15.66:0/2336334757 wait complete. 2024-09-10T22:35:15.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.2 2024-09-10T22:35:15.887 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.0 2024-09-10T22:35:15.893 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-09-10T22:35:15.893 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-10T22:35:15.895 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:15.895 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:35:15.895 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.038+0000 7f0c3d471640 1 -- 172.21.15.66:0/1238724031 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3815be30 msgr2=0x7f0c380688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.038+0000 7f0c3d471640 1 --2- 172.21.15.66:0/1238724031 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3815be30 0x7f0c380688e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009860 tx=0x7f0c2802a5a0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.039+0000 7f0c3d471640 1 -- 172.21.15.66:0/1238724031 shutdown_connections 2024-09-10T22:35:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.039+0000 7f0c3d471640 1 --2- 172.21.15.66:0/1238724031 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c3815be30 0x7f0c380688e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.039+0000 7f0c3d471640 1 -- 172.21.15.66:0/1238724031 >> 172.21.15.66:0/1238724031 conn(0x7f0c380d0430 msgr2=0x7f0c380d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.039+0000 7f0c3d471640 1 -- 172.21.15.66:0/1238724031 shutdown_connections 2024-09-10T22:35:16.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.039+0000 7f0c3d471640 1 -- 172.21.15.66:0/1238724031 wait complete. 2024-09-10T22:35:16.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.040+0000 7f0c3d471640 1 Processor -- start 2024-09-10T22:35:16.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.040+0000 7f0c3d471640 1 -- start start 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c380b1650 0x7f0c380c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c3815be30 0x7f0c381551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c38160350 0x7f0c38155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0c380cc590 con 0x7f0c3815be30 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c380b0d10 con 0x7f0c380b1650 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0c380cc710 con 0x7f0c38160350 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c3815be30 0x7f0c381551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c3815be30 0x7f0c381551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35922/0 (socket says 172.21.15.66:35922) 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 -- 172.21.15.66:0/755460103 learned_addr learned my addr 172.21.15.66:0/755460103 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:16.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c377fe640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c38160350 0x7f0c38155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c36ffd640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c380b1650 0x7f0c380c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 -- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c38160350 msgr2=0x7f0c38155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c38160350 0x7f0c38155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 -- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c380b1650 msgr2=0x7f0c380c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c380b1650 0x7f0c380c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 -- 172.21.15.66:0/755460103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c380ca410 con 0x7f0c3815be30 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c36ffd640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c380b1650 0x7f0c380c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c367fc640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c3815be30 0x7f0c381551d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c3c0 tx=0x7f0c2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c1f7fe640 1 -- 172.21.15.66:0/755460103 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c011790 con 0x7f0c3815be30 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c1f7fe640 1 -- 172.21.15.66:0/755460103 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c2c017070 con 0x7f0c3815be30 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c380b13f0 con 0x7f0c3815be30 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c1f7fe640 1 -- 172.21.15.66:0/755460103 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c011ce0 con 0x7f0c3815be30 2024-09-10T22:35:16.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.041+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c380b40b0 con 0x7f0c3815be30 2024-09-10T22:35:16.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.042+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c381600e0 con 0x7f0c3815be30 2024-09-10T22:35:16.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.043+0000 7f0c1f7fe640 1 -- 172.21.15.66:0/755460103 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0c2c01c460 con 0x7f0c3815be30 2024-09-10T22:35:16.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.044+0000 7f0c1f7fe640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0c24077f70 0x7f0c2407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.044+0000 7f0c1f7fe640 1 -- 172.21.15.66:0/755460103 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0c2c099ab0 con 0x7f0c3815be30 2024-09-10T22:35:16.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.044+0000 7f0c36ffd640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0c24077f70 0x7f0c2407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.044+0000 7f0c36ffd640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0c24077f70 0x7f0c2407a430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c28034000 tx=0x7f0c28004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:16.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.048+0000 7f0c1f7fe640 1 -- 172.21.15.66:0/755460103 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0c381600e0 con 0x7f0c3815be30 2024-09-10T22:35:16.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.232+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c3806d810 con 0x7f0c24077f70 2024-09-10T22:35:16.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.237+0000 7f0c1f7fe640 1 -- 172.21.15.66:0/755460103 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f0c3806d810 con 0x7f0c24077f70 2024-09-10T22:35:16.241 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:16.242 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0c24077f70 msgr2=0x7f0c2407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0c24077f70 0x7f0c2407a430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0c28034000 tx=0x7f0c28004730 comp rx=0 tx=0).stop 2024-09-10T22:35:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c3815be30 msgr2=0x7f0c381551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c3815be30 0x7f0c381551d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c3c0 tx=0x7f0c2c00c880 comp rx=0 tx=0).stop 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 shutdown_connections 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0c24077f70 0x7f0c2407a430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0c38160350 0x7f0c38155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c3815be30 0x7f0c381551d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.240+0000 7f0c3d471640 1 --2- 172.21.15.66:0/755460103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c380b1650 0x7f0c380c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.241+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 >> 172.21.15.66:0/755460103 conn(0x7f0c380d0430 msgr2=0x7f0c380ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.241+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 shutdown_connections 2024-09-10T22:35:16.244 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.241+0000 7f0c3d471640 1 -- 172.21.15.66:0/755460103 wait complete. 2024-09-10T22:35:16.252 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T22:35:14.595453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235760,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738960,"statfs":{"total":773094113280,"available":772852695040,"internally_reserved":0,"allocated":2924544,"data_stored":1764124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.535525"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283978+0000","last_change":"2024-09-10T22:35:03.064865+0000","last_active":"2024-09-10T22:35:04.283978+0000","last_peered":"2024-09-10T22:35:04.283978+0000","last_clean":"2024-09-10T22:35:04.283978+0000","last_became_active":"2024-09-10T22:35:03.064552+0000","last_became_peered":"2024-09-10T22:35:03.064552+0000","last_unstale":"2024-09-10T22:35:04.283978+0000","last_undegraded":"2024-09-10T22:35:04.283978+0000","last_fullsized":"2024-09-10T22:35:04.283978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:42.990367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060843+0000","last_change":"2024-09-10T22:35:03.065907+0000","last_active":"2024-09-10T22:35:04.060843+0000","last_peered":"2024-09-10T22:35:04.060843+0000","last_clean":"2024-09-10T22:35:04.060843+0000","last_became_active":"2024-09-10T22:35:03.065607+0000","last_became_peered":"2024-09-10T22:35:03.065607+0000","last_unstale":"2024-09-10T22:35:04.060843+0000","last_undegraded":"2024-09-10T22:35:04.060843+0000","last_fullsized":"2024-09-10T22:35:04.060843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283605+0000","last_change":"2024-09-10T22:35:03.063096+0000","last_active":"2024-09-10T22:35:04.283605+0000","last_peered":"2024-09-10T22:35:04.283605+0000","last_clean":"2024-09-10T22:35:04.283605+0000","last_became_active":"2024-09-10T22:35:03.057986+0000","last_became_peered":"2024-09-10T22:35:03.057986+0000","last_unstale":"2024-09-10T22:35:04.283605+0000","last_undegraded":"2024-09-10T22:35:04.283605+0000","last_fullsized":"2024-09-10T22:35:04.283605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:41.190200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062415+0000","last_change":"2024-09-10T22:35:03.076412+0000","last_active":"2024-09-10T22:35:04.062415+0000","last_peered":"2024-09-10T22:35:04.062415+0000","last_clean":"2024-09-10T22:35:04.062415+0000","last_became_active":"2024-09-10T22:35:03.076037+0000","last_became_peered":"2024-09-10T22:35:03.076037+0000","last_unstale":"2024-09-10T22:35:04.062415+0000","last_undegraded":"2024-09-10T22:35:04.062415+0000","last_fullsized":"2024-09-10T22:35:04.062415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:35.334336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284073+0000","last_change":"2024-09-10T22:35:03.061394+0000","last_active":"2024-09-10T22:35:04.284073+0000","last_peered":"2024-09-10T22:35:04.284073+0000","last_clean":"2024-09-10T22:35:04.284073+0000","last_became_active":"2024-09-10T22:35:03.061111+0000","last_became_peered":"2024-09-10T22:35:03.061111+0000","last_unstale":"2024-09-10T22:35:04.284073+0000","last_undegraded":"2024-09-10T22:35:04.284073+0000","last_fullsized":"2024-09-10T22:35:04.284073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:48.372055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062681+0000","last_change":"2024-09-10T22:35:03.062367+0000","last_active":"2024-09-10T22:35:04.062681+0000","last_peered":"2024-09-10T22:35:04.062681+0000","last_clean":"2024-09-10T22:35:04.062681+0000","last_became_active":"2024-09-10T22:35:03.061999+0000","last_became_peered":"2024-09-10T22:35:03.061999+0000","last_unstale":"2024-09-10T22:35:04.062681+0000","last_undegraded":"2024-09-10T22:35:04.062681+0000","last_fullsized":"2024-09-10T22:35:04.062681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:42.990367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282913+0000","last_change":"2024-09-10T22:35:03.065678+0000","last_active":"2024-09-10T22:35:04.282913+0000","last_peered":"2024-09-10T22:35:04.282913+0000","last_clean":"2024-09-10T22:35:04.282913+0000","last_became_active":"2024-09-10T22:35:03.065414+0000","last_became_peered":"2024-09-10T22:35:03.065414+0000","last_unstale":"2024-09-10T22:35:04.282913+0000","last_undegraded":"2024-09-10T22:35:04.282913+0000","last_fullsized":"2024-09-10T22:35:04.282913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:56.275057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283932+0000","last_change":"2024-09-10T22:35:03.072149+0000","last_active":"2024-09-10T22:35:04.283932+0000","last_peered":"2024-09-10T22:35:04.283932+0000","last_clean":"2024-09-10T22:35:04.283932+0000","last_became_active":"2024-09-10T22:35:03.071819+0000","last_became_peered":"2024-09-10T22:35:03.071819+0000","last_unstale":"2024-09-10T22:35:04.283932+0000","last_undegraded":"2024-09-10T22:35:04.283932+0000","last_fullsized":"2024-09-10T22:35:04.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:26.466821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284074+0000","last_change":"2024-09-10T22:35:03.068359+0000","last_active":"2024-09-10T22:35:04.284074+0000","last_peered":"2024-09-10T22:35:04.284074+0000","last_clean":"2024-09-10T22:35:04.284074+0000","last_became_active":"2024-09-10T22:35:03.067889+0000","last_became_peered":"2024-09-10T22:35:03.067889+0000","last_unstale":"2024-09-10T22:35:04.284074+0000","last_undegraded":"2024-09-10T22:35:04.284074+0000","last_fullsized":"2024-09-10T22:35:04.284074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:06.735630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283225+0000","last_change":"2024-09-10T22:35:03.061509+0000","last_active":"2024-09-10T22:35:04.283225+0000","last_peered":"2024-09-10T22:35:04.283225+0000","last_clean":"2024-09-10T22:35:04.283225+0000","last_became_active":"2024-09-10T22:35:03.061170+0000","last_became_peered":"2024-09-10T22:35:03.061170+0000","last_unstale":"2024-09-10T22:35:04.283225+0000","last_undegraded":"2024-09-10T22:35:04.283225+0000","last_fullsized":"2024-09-10T22:35:04.283225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:26.993716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283408+0000","last_change":"2024-09-10T22:35:03.061387+0000","last_active":"2024-09-10T22:35:04.283408+0000","last_peered":"2024-09-10T22:35:04.283408+0000","last_clean":"2024-09-10T22:35:04.283408+0000","last_became_active":"2024-09-10T22:35:03.061088+0000","last_became_peered":"2024-09-10T22:35:03.061088+0000","last_unstale":"2024-09-10T22:35:04.283408+0000","last_undegraded":"2024-09-10T22:35:04.283408+0000","last_fullsized":"2024-09-10T22:35:04.283408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:31.398073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063686+0000","last_change":"2024-09-10T22:35:03.058702+0000","last_active":"2024-09-10T22:35:04.063686+0000","last_peered":"2024-09-10T22:35:04.063686+0000","last_clean":"2024-09-10T22:35:04.063686+0000","last_became_active":"2024-09-10T22:35:03.058419+0000","last_became_peered":"2024-09-10T22:35:03.058419+0000","last_unstale":"2024-09-10T22:35:04.063686+0000","last_undegraded":"2024-09-10T22:35:04.063686+0000","last_fullsized":"2024-09-10T22:35:04.063686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:59.284853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283542+0000","last_change":"2024-09-10T22:35:03.080808+0000","last_active":"2024-09-10T22:35:04.283542+0000","last_peered":"2024-09-10T22:35:04.283542+0000","last_clean":"2024-09-10T22:35:04.283542+0000","last_became_active":"2024-09-10T22:35:03.080467+0000","last_became_peered":"2024-09-10T22:35:03.080467+0000","last_unstale":"2024-09-10T22:35:04.283542+0000","last_undegraded":"2024-09-10T22:35:04.283542+0000","last_fullsized":"2024-09-10T22:35:04.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282815+0000","last_change":"2024-09-10T22:35:03.057789+0000","last_active":"2024-09-10T22:35:04.282815+0000","last_peered":"2024-09-10T22:35:04.282815+0000","last_clean":"2024-09-10T22:35:04.282815+0000","last_became_active":"2024-09-10T22:35:03.057450+0000","last_became_peered":"2024-09-10T22:35:03.057450+0000","last_unstale":"2024-09-10T22:35:04.282815+0000","last_undegraded":"2024-09-10T22:35:04.282815+0000","last_fullsized":"2024-09-10T22:35:04.282815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:30.343040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060665+0000","last_change":"2024-09-10T22:35:03.062056+0000","last_active":"2024-09-10T22:35:04.060665+0000","last_peered":"2024-09-10T22:35:04.060665+0000","last_clean":"2024-09-10T22:35:04.060665+0000","last_became_active":"2024-09-10T22:35:03.061769+0000","last_became_peered":"2024-09-10T22:35:03.061769+0000","last_unstale":"2024-09-10T22:35:04.060665+0000","last_undegraded":"2024-09-10T22:35:04.060665+0000","last_fullsized":"2024-09-10T22:35:04.060665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:02.406217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.061953+0000","last_change":"2024-09-10T22:35:03.067251+0000","last_active":"2024-09-10T22:35:04.061953+0000","last_peered":"2024-09-10T22:35:04.061953+0000","last_clean":"2024-09-10T22:35:04.061953+0000","last_became_active":"2024-09-10T22:35:03.066980+0000","last_became_peered":"2024-09-10T22:35:03.066980+0000","last_unstale":"2024-09-10T22:35:04.061953+0000","last_undegraded":"2024-09-10T22:35:04.061953+0000","last_fullsized":"2024-09-10T22:35:04.061953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:45.349941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062881+0000","last_change":"2024-09-10T22:35:03.070627+0000","last_active":"2024-09-10T22:35:04.062881+0000","last_peered":"2024-09-10T22:35:04.062881+0000","last_clean":"2024-09-10T22:35:04.062881+0000","last_became_active":"2024-09-10T22:35:03.070365+0000","last_became_peered":"2024-09-10T22:35:03.070365+0000","last_unstale":"2024-09-10T22:35:04.062881+0000","last_undegraded":"2024-09-10T22:35:04.062881+0000","last_fullsized":"2024-09-10T22:35:04.062881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:56.758850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282700+0000","last_change":"2024-09-10T22:35:03.060118+0000","last_active":"2024-09-10T22:35:04.282700+0000","last_peered":"2024-09-10T22:35:04.282700+0000","last_clean":"2024-09-10T22:35:04.282700+0000","last_became_active":"2024-09-10T22:35:03.059799+0000","last_became_peered":"2024-09-10T22:35:03.059799+0000","last_unstale":"2024-09-10T22:35:04.282700+0000","last_undegraded":"2024-09-10T22:35:04.282700+0000","last_fullsized":"2024-09-10T22:35:04.282700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:50.965802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284045+0000","last_change":"2024-09-10T22:35:03.062861+0000","last_active":"2024-09-10T22:35:04.284045+0000","last_peered":"2024-09-10T22:35:04.284045+0000","last_clean":"2024-09-10T22:35:04.284045+0000","last_became_active":"2024-09-10T22:35:03.062586+0000","last_became_peered":"2024-09-10T22:35:03.062586+0000","last_unstale":"2024-09-10T22:35:04.284045+0000","last_undegraded":"2024-09-10T22:35:04.284045+0000","last_fullsized":"2024-09-10T22:35:04.284045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:22.142634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283055+0000","last_change":"2024-09-10T22:35:03.070244+0000","last_active":"2024-09-10T22:35:04.283055+0000","last_peered":"2024-09-10T22:35:04.283055+0000","last_clean":"2024-09-10T22:35:04.283055+0000","last_became_active":"2024-09-10T22:35:03.067927+0000","last_became_peered":"2024-09-10T22:35:03.067927+0000","last_unstale":"2024-09-10T22:35:04.283055+0000","last_undegraded":"2024-09-10T22:35:04.283055+0000","last_fullsized":"2024-09-10T22:35:04.283055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:51.838173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283820+0000","last_change":"2024-09-10T22:35:03.065175+0000","last_active":"2024-09-10T22:35:04.283820+0000","last_peered":"2024-09-10T22:35:04.283820+0000","last_clean":"2024-09-10T22:35:04.283820+0000","last_became_active":"2024-09-10T22:35:03.064880+0000","last_became_peered":"2024-09-10T22:35:03.064880+0000","last_unstale":"2024-09-10T22:35:04.283820+0000","last_undegraded":"2024-09-10T22:35:04.283820+0000","last_fullsized":"2024-09-10T22:35:04.283820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:34.515907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062375+0000","last_change":"2024-09-10T22:35:03.069577+0000","last_active":"2024-09-10T22:35:04.062375+0000","last_peered":"2024-09-10T22:35:04.062375+0000","last_clean":"2024-09-10T22:35:04.062375+0000","last_became_active":"2024-09-10T22:35:03.069224+0000","last_became_peered":"2024-09-10T22:35:03.069224+0000","last_unstale":"2024-09-10T22:35:04.062375+0000","last_undegraded":"2024-09-10T22:35:04.062375+0000","last_fullsized":"2024-09-10T22:35:04.062375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:16.644778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283468+0000","last_change":"2024-09-10T22:35:03.076617+0000","last_active":"2024-09-10T22:35:04.283468+0000","last_peered":"2024-09-10T22:35:04.283468+0000","last_clean":"2024-09-10T22:35:04.283468+0000","last_became_active":"2024-09-10T22:35:03.075527+0000","last_became_peered":"2024-09-10T22:35:03.075527+0000","last_unstale":"2024-09-10T22:35:04.283468+0000","last_undegraded":"2024-09-10T22:35:04.283468+0000","last_fullsized":"2024-09-10T22:35:04.283468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062216+0000","last_change":"2024-09-10T22:35:03.072643+0000","last_active":"2024-09-10T22:35:04.062216+0000","last_peered":"2024-09-10T22:35:04.062216+0000","last_clean":"2024-09-10T22:35:04.062216+0000","last_became_active":"2024-09-10T22:35:03.072354+0000","last_became_peered":"2024-09-10T22:35:03.072354+0000","last_unstale":"2024-09-10T22:35:04.062216+0000","last_undegraded":"2024-09-10T22:35:04.062216+0000","last_fullsized":"2024-09-10T22:35:04.062216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:25.832743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283390+0000","last_change":"2024-09-10T22:35:03.076662+0000","last_active":"2024-09-10T22:35:04.283390+0000","last_peered":"2024-09-10T22:35:04.283390+0000","last_clean":"2024-09-10T22:35:04.283390+0000","last_became_active":"2024-09-10T22:35:03.075696+0000","last_became_peered":"2024-09-10T22:35:03.075696+0000","last_unstale":"2024-09-10T22:35:04.283390+0000","last_undegraded":"2024-09-10T22:35:04.283390+0000","last_fullsized":"2024-09-10T22:35:04.283390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:59.070897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283416+0000","last_change":"2024-09-10T22:35:03.070886+0000","last_active":"2024-09-10T22:35:04.283416+0000","last_peered":"2024-09-10T22:35:04.283416+0000","last_clean":"2024-09-10T22:35:04.283416+0000","last_became_active":"2024-09-10T22:35:03.070569+0000","last_became_peered":"2024-09-10T22:35:03.070569+0000","last_unstale":"2024-09-10T22:35:04.283416+0000","last_undegraded":"2024-09-10T22:35:04.283416+0000","last_fullsized":"2024-09-10T22:35:04.283416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:25.138442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060768+0000","last_change":"2024-09-10T22:35:03.074633+0000","last_active":"2024-09-10T22:35:04.060768+0000","last_peered":"2024-09-10T22:35:04.060768+0000","last_clean":"2024-09-10T22:35:04.060768+0000","last_became_active":"2024-09-10T22:35:03.074332+0000","last_became_peered":"2024-09-10T22:35:03.074332+0000","last_unstale":"2024-09-10T22:35:04.060768+0000","last_undegraded":"2024-09-10T22:35:04.060768+0000","last_fullsized":"2024-09-10T22:35:04.060768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283311+0000","last_change":"2024-09-10T22:35:03.077344+0000","last_active":"2024-09-10T22:35:04.283311+0000","last_peered":"2024-09-10T22:35:04.283311+0000","last_clean":"2024-09-10T22:35:04.283311+0000","last_became_active":"2024-09-10T22:35:03.075668+0000","last_became_peered":"2024-09-10T22:35:03.075668+0000","last_unstale":"2024-09-10T22:35:04.283311+0000","last_undegraded":"2024-09-10T22:35:04.283311+0000","last_fullsized":"2024-09-10T22:35:04.283311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283878+0000","last_change":"2024-09-10T22:35:03.078565+0000","last_active":"2024-09-10T22:35:04.283878+0000","last_peered":"2024-09-10T22:35:04.283878+0000","last_clean":"2024-09-10T22:35:04.283878+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.283878+0000","last_undegraded":"2024-09-10T22:35:04.283878+0000","last_fullsized":"2024-09-10T22:35:04.283878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:57.500600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062623+0000","last_change":"2024-09-10T22:35:03.075919+0000","last_active":"2024-09-10T22:35:04.062623+0000","last_peered":"2024-09-10T22:35:04.062623+0000","last_clean":"2024-09-10T22:35:04.062623+0000","last_became_active":"2024-09-10T22:35:03.075584+0000","last_became_peered":"2024-09-10T22:35:03.075584+0000","last_unstale":"2024-09-10T22:35:04.062623+0000","last_undegraded":"2024-09-10T22:35:04.062623+0000","last_fullsized":"2024-09-10T22:35:04.062623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:57.797898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282910+0000","last_change":"2024-09-10T22:35:03.058708+0000","last_active":"2024-09-10T22:35:04.282910+0000","last_peered":"2024-09-10T22:35:04.282910+0000","last_clean":"2024-09-10T22:35:04.282910+0000","last_became_active":"2024-09-10T22:35:03.058399+0000","last_became_peered":"2024-09-10T22:35:03.058399+0000","last_unstale":"2024-09-10T22:35:04.282910+0000","last_undegraded":"2024-09-10T22:35:04.282910+0000","last_fullsized":"2024-09-10T22:35:04.282910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:21.971330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282922+0000","last_change":"2024-09-10T22:35:03.066152+0000","last_active":"2024-09-10T22:35:04.282922+0000","last_peered":"2024-09-10T22:35:04.282922+0000","last_clean":"2024-09-10T22:35:04.282922+0000","last_became_active":"2024-09-10T22:35:03.065897+0000","last_became_peered":"2024-09-10T22:35:03.065897+0000","last_unstale":"2024-09-10T22:35:04.282922+0000","last_undegraded":"2024-09-10T22:35:04.282922+0000","last_fullsized":"2024-09-10T22:35:04.282922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:01.896261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063193+0000","last_change":"2024-09-10T22:35:03.062529+0000","last_active":"2024-09-10T22:35:04.063193+0000","last_peered":"2024-09-10T22:35:04.063193+0000","last_clean":"2024-09-10T22:35:04.063193+0000","last_became_active":"2024-09-10T22:35:03.059720+0000","last_became_peered":"2024-09-10T22:35:03.059720+0000","last_unstale":"2024-09-10T22:35:04.063193+0000","last_undegraded":"2024-09-10T22:35:04.063193+0000","last_fullsized":"2024-09-10T22:35:04.063193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:45.451161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283873+0000","last_change":"2024-09-10T22:35:02.060438+0000","last_active":"2024-09-10T22:35:04.283873+0000","last_peered":"2024-09-10T22:35:04.283873+0000","last_clean":"2024-09-10T22:35:04.283873+0000","last_became_active":"2024-09-10T22:35:02.060124+0000","last_became_peered":"2024-09-10T22:35:02.060124+0000","last_unstale":"2024-09-10T22:35:04.283873+0000","last_undegraded":"2024-09-10T22:35:04.283873+0000","last_fullsized":"2024-09-10T22:35:04.283873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283981+0000","last_change":"2024-09-10T22:35:03.068471+0000","last_active":"2024-09-10T22:35:04.283981+0000","last_peered":"2024-09-10T22:35:04.283981+0000","last_clean":"2024-09-10T22:35:04.283981+0000","last_became_active":"2024-09-10T22:35:03.068079+0000","last_became_peered":"2024-09-10T22:35:03.068079+0000","last_unstale":"2024-09-10T22:35:04.283981+0000","last_undegraded":"2024-09-10T22:35:04.283981+0000","last_fullsized":"2024-09-10T22:35:04.283981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:53.676311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283156+0000","last_change":"2024-09-10T22:35:02.060297+0000","last_active":"2024-09-10T22:35:04.283156+0000","last_peered":"2024-09-10T22:35:04.283156+0000","last_clean":"2024-09-10T22:35:04.283156+0000","last_became_active":"2024-09-10T22:35:02.059966+0000","last_became_peered":"2024-09-10T22:35:02.059966+0000","last_unstale":"2024-09-10T22:35:04.283156+0000","last_undegraded":"2024-09-10T22:35:04.283156+0000","last_fullsized":"2024-09-10T22:35:04.283156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.073637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283633+0000","last_change":"2024-09-10T22:35:02.058735+0000","last_active":"2024-09-10T22:35:04.283633+0000","last_peered":"2024-09-10T22:35:04.283633+0000","last_clean":"2024-09-10T22:35:04.283633+0000","last_became_active":"2024-09-10T22:35:02.058425+0000","last_became_peered":"2024-09-10T22:35:02.058425+0000","last_unstale":"2024-09-10T22:35:04.283633+0000","last_undegraded":"2024-09-10T22:35:04.283633+0000","last_fullsized":"2024-09-10T22:35:04.283633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284127+0000","last_change":"2024-09-10T22:35:03.070043+0000","last_active":"2024-09-10T22:35:04.284127+0000","last_peered":"2024-09-10T22:35:04.284127+0000","last_clean":"2024-09-10T22:35:04.284127+0000","last_became_active":"2024-09-10T22:35:03.069418+0000","last_became_peered":"2024-09-10T22:35:03.069418+0000","last_unstale":"2024-09-10T22:35:04.284127+0000","last_undegraded":"2024-09-10T22:35:04.284127+0000","last_fullsized":"2024-09-10T22:35:04.284127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:23.917793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062027+0000","last_change":"2024-09-10T22:35:02.056871+0000","last_active":"2024-09-10T22:35:04.062027+0000","last_peered":"2024-09-10T22:35:04.062027+0000","last_clean":"2024-09-10T22:35:04.062027+0000","last_became_active":"2024-09-10T22:35:02.056541+0000","last_became_peered":"2024-09-10T22:35:02.056541+0000","last_unstale":"2024-09-10T22:35:04.062027+0000","last_undegraded":"2024-09-10T22:35:04.062027+0000","last_fullsized":"2024-09-10T22:35:04.062027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:58.279821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060647+0000","last_change":"2024-09-10T22:35:03.074156+0000","last_active":"2024-09-10T22:35:04.060647+0000","last_peered":"2024-09-10T22:35:04.060647+0000","last_clean":"2024-09-10T22:35:04.060647+0000","last_became_active":"2024-09-10T22:35:03.073836+0000","last_became_peered":"2024-09-10T22:35:03.073836+0000","last_unstale":"2024-09-10T22:35:04.060647+0000","last_undegraded":"2024-09-10T22:35:04.060647+0000","last_fullsized":"2024-09-10T22:35:04.060647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283581+0000","last_change":"2024-09-10T22:35:02.063854+0000","last_active":"2024-09-10T22:35:04.283581+0000","last_peered":"2024-09-10T22:35:04.283581+0000","last_clean":"2024-09-10T22:35:04.283581+0000","last_became_active":"2024-09-10T22:35:02.063214+0000","last_became_peered":"2024-09-10T22:35:02.063214+0000","last_unstale":"2024-09-10T22:35:04.283581+0000","last_undegraded":"2024-09-10T22:35:04.283581+0000","last_fullsized":"2024-09-10T22:35:04.283581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:23.311121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062461+0000","last_change":"2024-09-10T22:35:03.069518+0000","last_active":"2024-09-10T22:35:04.062461+0000","last_peered":"2024-09-10T22:35:04.062461+0000","last_clean":"2024-09-10T22:35:04.062461+0000","last_became_active":"2024-09-10T22:35:03.069271+0000","last_became_peered":"2024-09-10T22:35:03.069271+0000","last_unstale":"2024-09-10T22:35:04.062461+0000","last_undegraded":"2024-09-10T22:35:04.062461+0000","last_fullsized":"2024-09-10T22:35:04.062461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:28.595416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062604+0000","last_change":"2024-09-10T22:35:02.060887+0000","last_active":"2024-09-10T22:35:04.062604+0000","last_peered":"2024-09-10T22:35:04.062604+0000","last_clean":"2024-09-10T22:35:04.062604+0000","last_became_active":"2024-09-10T22:35:02.060054+0000","last_became_peered":"2024-09-10T22:35:02.060054+0000","last_unstale":"2024-09-10T22:35:04.062604+0000","last_undegraded":"2024-09-10T22:35:04.062604+0000","last_fullsized":"2024-09-10T22:35:04.062604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:32.374399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283914+0000","last_change":"2024-09-10T22:35:03.063406+0000","last_active":"2024-09-10T22:35:04.283914+0000","last_peered":"2024-09-10T22:35:04.283914+0000","last_clean":"2024-09-10T22:35:04.283914+0000","last_became_active":"2024-09-10T22:35:03.062960+0000","last_became_peered":"2024-09-10T22:35:03.062960+0000","last_unstale":"2024-09-10T22:35:04.283914+0000","last_undegraded":"2024-09-10T22:35:04.283914+0000","last_fullsized":"2024-09-10T22:35:04.283914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:08.845668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060881+0000","last_change":"2024-09-10T22:35:02.061366+0000","last_active":"2024-09-10T22:35:04.060881+0000","last_peered":"2024-09-10T22:35:04.060881+0000","last_clean":"2024-09-10T22:35:04.060881+0000","last_became_active":"2024-09-10T22:35:02.061092+0000","last_became_peered":"2024-09-10T22:35:02.061092+0000","last_unstale":"2024-09-10T22:35:04.060881+0000","last_undegraded":"2024-09-10T22:35:04.060881+0000","last_fullsized":"2024-09-10T22:35:04.060881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063828+0000","last_change":"2024-09-10T22:35:03.066189+0000","last_active":"2024-09-10T22:35:04.063828+0000","last_peered":"2024-09-10T22:35:04.063828+0000","last_clean":"2024-09-10T22:35:04.063828+0000","last_became_active":"2024-09-10T22:35:03.064797+0000","last_became_peered":"2024-09-10T22:35:03.064797+0000","last_unstale":"2024-09-10T22:35:04.063828+0000","last_undegraded":"2024-09-10T22:35:04.063828+0000","last_fullsized":"2024-09-10T22:35:04.063828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:33.379430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283111+0000","last_change":"2024-09-10T22:35:02.056652+0000","last_active":"2024-09-10T22:35:04.283111+0000","last_peered":"2024-09-10T22:35:04.283111+0000","last_clean":"2024-09-10T22:35:04.283111+0000","last_became_active":"2024-09-10T22:35:02.056381+0000","last_became_peered":"2024-09-10T22:35:02.056381+0000","last_unstale":"2024-09-10T22:35:04.283111+0000","last_undegraded":"2024-09-10T22:35:04.283111+0000","last_fullsized":"2024-09-10T22:35:04.283111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:42.884854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283576+0000","last_change":"2024-09-10T22:35:03.071792+0000","last_active":"2024-09-10T22:35:04.283576+0000","last_peered":"2024-09-10T22:35:04.283576+0000","last_clean":"2024-09-10T22:35:04.283576+0000","last_became_active":"2024-09-10T22:35:03.071456+0000","last_became_peered":"2024-09-10T22:35:03.071456+0000","last_unstale":"2024-09-10T22:35:04.283576+0000","last_undegraded":"2024-09-10T22:35:04.283576+0000","last_fullsized":"2024-09-10T22:35:04.283576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:35.334336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283948+0000","last_change":"2024-09-10T22:35:02.062689+0000","last_active":"2024-09-10T22:35:04.283948+0000","last_peered":"2024-09-10T22:35:04.283948+0000","last_clean":"2024-09-10T22:35:04.283948+0000","last_became_active":"2024-09-10T22:35:02.062395+0000","last_became_peered":"2024-09-10T22:35:02.062395+0000","last_unstale":"2024-09-10T22:35:04.283948+0000","last_undegraded":"2024-09-10T22:35:04.283948+0000","last_fullsized":"2024-09-10T22:35:04.283948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:10.653575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283121+0000","last_change":"2024-09-10T22:35:03.062742+0000","last_active":"2024-09-10T22:35:04.283121+0000","last_peered":"2024-09-10T22:35:04.283121+0000","last_clean":"2024-09-10T22:35:04.283121+0000","last_became_active":"2024-09-10T22:35:03.062374+0000","last_became_peered":"2024-09-10T22:35:03.062374+0000","last_unstale":"2024-09-10T22:35:04.283121+0000","last_undegraded":"2024-09-10T22:35:04.283121+0000","last_fullsized":"2024-09-10T22:35:04.283121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:26.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062695+0000","last_change":"2024-09-10T22:35:02.059477+0000","last_active":"2024-09-10T22:35:04.062695+0000","last_peered":"2024-09-10T22:35:04.062695+0000","last_clean":"2024-09-10T22:35:04.062695+0000","last_became_active":"2024-09-10T22:35:02.058863+0000","last_became_peered":"2024-09-10T22:35:02.058863+0000","last_unstale":"2024-09-10T22:35:04.062695+0000","last_undegraded":"2024-09-10T22:35:04.062695+0000","last_fullsized":"2024-09-10T22:35:04.062695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:44.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284043+0000","last_change":"2024-09-10T22:35:03.073670+0000","last_active":"2024-09-10T22:35:04.284043+0000","last_peered":"2024-09-10T22:35:04.284043+0000","last_clean":"2024-09-10T22:35:04.284043+0000","last_became_active":"2024-09-10T22:35:03.073362+0000","last_became_peered":"2024-09-10T22:35:03.073362+0000","last_unstale":"2024-09-10T22:35:04.284043+0000","last_undegraded":"2024-09-10T22:35:04.284043+0000","last_fullsized":"2024-09-10T22:35:04.284043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:56.758850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283238+0000","last_change":"2024-09-10T22:35:02.060433+0000","last_active":"2024-09-10T22:35:04.283238+0000","last_peered":"2024-09-10T22:35:04.283238+0000","last_clean":"2024-09-10T22:35:04.283238+0000","last_became_active":"2024-09-10T22:35:02.059798+0000","last_became_peered":"2024-09-10T22:35:02.059798+0000","last_unstale":"2024-09-10T22:35:04.283238+0000","last_undegraded":"2024-09-10T22:35:04.283238+0000","last_fullsized":"2024-09-10T22:35:04.283238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:07.281434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283176+0000","last_change":"2024-09-10T22:35:03.065328+0000","last_active":"2024-09-10T22:35:04.283176+0000","last_peered":"2024-09-10T22:35:04.283176+0000","last_clean":"2024-09-10T22:35:04.283176+0000","last_became_active":"2024-09-10T22:35:03.065050+0000","last_became_peered":"2024-09-10T22:35:03.065050+0000","last_unstale":"2024-09-10T22:35:04.283176+0000","last_undegraded":"2024-09-10T22:35:04.283176+0000","last_fullsized":"2024-09-10T22:35:04.283176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:28.267375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282601+0000","last_change":"2024-09-10T22:35:02.058914+0000","last_active":"2024-09-10T22:35:04.282601+0000","last_peered":"2024-09-10T22:35:04.282601+0000","last_clean":"2024-09-10T22:35:04.282601+0000","last_became_active":"2024-09-10T22:35:02.058054+0000","last_became_peered":"2024-09-10T22:35:02.058054+0000","last_unstale":"2024-09-10T22:35:04.282601+0000","last_undegraded":"2024-09-10T22:35:04.282601+0000","last_fullsized":"2024-09-10T22:35:04.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:07.958631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283220+0000","last_change":"2024-09-10T22:35:03.074096+0000","last_active":"2024-09-10T22:35:04.283220+0000","last_peered":"2024-09-10T22:35:04.283220+0000","last_clean":"2024-09-10T22:35:04.283220+0000","last_became_active":"2024-09-10T22:35:03.073454+0000","last_became_peered":"2024-09-10T22:35:03.073454+0000","last_unstale":"2024-09-10T22:35:04.283220+0000","last_undegraded":"2024-09-10T22:35:04.283220+0000","last_fullsized":"2024-09-10T22:35:04.283220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:16.644778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063831+0000","last_change":"2024-09-10T22:35:02.059067+0000","last_active":"2024-09-10T22:35:04.063831+0000","last_peered":"2024-09-10T22:35:04.063831+0000","last_clean":"2024-09-10T22:35:04.063831+0000","last_became_active":"2024-09-10T22:35:02.058800+0000","last_became_peered":"2024-09-10T22:35:02.058800+0000","last_unstale":"2024-09-10T22:35:04.063831+0000","last_undegraded":"2024-09-10T22:35:04.063831+0000","last_fullsized":"2024-09-10T22:35:04.063831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:32.586124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283832+0000","last_change":"2024-09-10T22:35:03.066866+0000","last_active":"2024-09-10T22:35:04.283832+0000","last_peered":"2024-09-10T22:35:04.283832+0000","last_clean":"2024-09-10T22:35:04.283832+0000","last_became_active":"2024-09-10T22:35:03.066570+0000","last_became_peered":"2024-09-10T22:35:03.066570+0000","last_unstale":"2024-09-10T22:35:04.283832+0000","last_undegraded":"2024-09-10T22:35:04.283832+0000","last_fullsized":"2024-09-10T22:35:04.283832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:57.797898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282840+0000","last_change":"2024-09-10T22:35:02.056593+0000","last_active":"2024-09-10T22:35:04.282840+0000","last_peered":"2024-09-10T22:35:04.282840+0000","last_clean":"2024-09-10T22:35:04.282840+0000","last_became_active":"2024-09-10T22:35:02.056253+0000","last_became_peered":"2024-09-10T22:35:02.056253+0000","last_unstale":"2024-09-10T22:35:04.282840+0000","last_undegraded":"2024-09-10T22:35:04.282840+0000","last_fullsized":"2024-09-10T22:35:04.282840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284206+0000","last_change":"2024-09-10T22:35:03.073877+0000","last_active":"2024-09-10T22:35:04.284206+0000","last_peered":"2024-09-10T22:35:04.284206+0000","last_clean":"2024-09-10T22:35:04.284206+0000","last_became_active":"2024-09-10T22:35:03.073479+0000","last_became_peered":"2024-09-10T22:35:03.073479+0000","last_unstale":"2024-09-10T22:35:04.284206+0000","last_undegraded":"2024-09-10T22:35:04.284206+0000","last_fullsized":"2024-09-10T22:35:04.284206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:31.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060976+0000","last_change":"2024-09-10T22:35:02.059186+0000","last_active":"2024-09-10T22:35:04.060976+0000","last_peered":"2024-09-10T22:35:04.060976+0000","last_clean":"2024-09-10T22:35:04.060976+0000","last_became_active":"2024-09-10T22:35:02.058905+0000","last_became_peered":"2024-09-10T22:35:02.058905+0000","last_unstale":"2024-09-10T22:35:04.060976+0000","last_undegraded":"2024-09-10T22:35:04.060976+0000","last_fullsized":"2024-09-10T22:35:04.060976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062968+0000","last_change":"2024-09-10T22:35:03.076118+0000","last_active":"2024-09-10T22:35:04.062968+0000","last_peered":"2024-09-10T22:35:04.062968+0000","last_clean":"2024-09-10T22:35:04.062968+0000","last_became_active":"2024-09-10T22:35:03.075621+0000","last_became_peered":"2024-09-10T22:35:03.075621+0000","last_unstale":"2024-09-10T22:35:04.062968+0000","last_undegraded":"2024-09-10T22:35:04.062968+0000","last_fullsized":"2024-09-10T22:35:04.062968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:48.372055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283210+0000","last_change":"2024-09-10T22:35:02.059788+0000","last_active":"2024-09-10T22:35:04.283210+0000","last_peered":"2024-09-10T22:35:04.283210+0000","last_clean":"2024-09-10T22:35:04.283210+0000","last_became_active":"2024-09-10T22:35:02.059419+0000","last_became_peered":"2024-09-10T22:35:02.059419+0000","last_unstale":"2024-09-10T22:35:04.283210+0000","last_undegraded":"2024-09-10T22:35:04.283210+0000","last_fullsized":"2024-09-10T22:35:04.283210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283722+0000","last_change":"2024-09-10T22:35:03.069011+0000","last_active":"2024-09-10T22:35:04.283722+0000","last_peered":"2024-09-10T22:35:04.283722+0000","last_clean":"2024-09-10T22:35:04.283722+0000","last_became_active":"2024-09-10T22:35:03.068736+0000","last_became_peered":"2024-09-10T22:35:03.068736+0000","last_unstale":"2024-09-10T22:35:04.283722+0000","last_undegraded":"2024-09-10T22:35:04.283722+0000","last_fullsized":"2024-09-10T22:35:04.283722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:28.595416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282999+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:04.282999+0000","last_peered":"2024-09-10T22:35:04.282999+0000","last_clean":"2024-09-10T22:35:04.282999+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:04.282999+0000","last_undegraded":"2024-09-10T22:35:04.282999+0000","last_fullsized":"2024-09-10T22:35:04.282999+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.061890+0000","last_change":"2024-09-10T22:35:02.057812+0000","last_active":"2024-09-10T22:35:04.061890+0000","last_peered":"2024-09-10T22:35:04.061890+0000","last_clean":"2024-09-10T22:35:04.061890+0000","last_became_active":"2024-09-10T22:35:02.057415+0000","last_became_peered":"2024-09-10T22:35:02.057415+0000","last_unstale":"2024-09-10T22:35:04.061890+0000","last_undegraded":"2024-09-10T22:35:04.061890+0000","last_fullsized":"2024-09-10T22:35:04.061890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:28.670744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283212+0000","last_change":"2024-09-10T22:35:03.079281+0000","last_active":"2024-09-10T22:35:04.283212+0000","last_peered":"2024-09-10T22:35:04.283212+0000","last_clean":"2024-09-10T22:35:04.283212+0000","last_became_active":"2024-09-10T22:35:03.078916+0000","last_became_peered":"2024-09-10T22:35:03.078916+0000","last_unstale":"2024-09-10T22:35:04.283212+0000","last_undegraded":"2024-09-10T22:35:04.283212+0000","last_fullsized":"2024-09-10T22:35:04.283212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282681+0000","last_change":"2024-09-10T22:35:02.065966+0000","last_active":"2024-09-10T22:35:04.282681+0000","last_peered":"2024-09-10T22:35:04.282681+0000","last_clean":"2024-09-10T22:35:04.282681+0000","last_became_active":"2024-09-10T22:35:02.065658+0000","last_became_peered":"2024-09-10T22:35:02.065658+0000","last_unstale":"2024-09-10T22:35:04.282681+0000","last_undegraded":"2024-09-10T22:35:04.282681+0000","last_fullsized":"2024-09-10T22:35:04.282681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:39.077105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282773+0000","last_change":"2024-09-10T22:35:03.055020+0000","last_active":"2024-09-10T22:35:04.282773+0000","last_peered":"2024-09-10T22:35:04.282773+0000","last_clean":"2024-09-10T22:35:04.282773+0000","last_became_active":"2024-09-10T22:35:03.052854+0000","last_became_peered":"2024-09-10T22:35:03.052854+0000","last_unstale":"2024-09-10T22:35:04.282773+0000","last_undegraded":"2024-09-10T22:35:04.282773+0000","last_fullsized":"2024-09-10T22:35:04.282773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:40.082137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283032+0000","last_change":"2024-09-10T22:35:02.057593+0000","last_active":"2024-09-10T22:35:04.283032+0000","last_peered":"2024-09-10T22:35:04.283032+0000","last_clean":"2024-09-10T22:35:04.283032+0000","last_became_active":"2024-09-10T22:35:02.057203+0000","last_became_peered":"2024-09-10T22:35:02.057203+0000","last_unstale":"2024-09-10T22:35:04.283032+0000","last_undegraded":"2024-09-10T22:35:04.283032+0000","last_fullsized":"2024-09-10T22:35:04.283032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:49.760894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060660+0000","last_change":"2024-09-10T22:35:03.057957+0000","last_active":"2024-09-10T22:35:04.060660+0000","last_peered":"2024-09-10T22:35:04.060660+0000","last_clean":"2024-09-10T22:35:04.060660+0000","last_became_active":"2024-09-10T22:35:03.057658+0000","last_became_peered":"2024-09-10T22:35:03.057658+0000","last_unstale":"2024-09-10T22:35:04.060660+0000","last_undegraded":"2024-09-10T22:35:04.060660+0000","last_fullsized":"2024-09-10T22:35:04.060660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062941+0000","last_change":"2024-09-10T22:35:02.059793+0000","last_active":"2024-09-10T22:35:04.062941+0000","last_peered":"2024-09-10T22:35:04.062941+0000","last_clean":"2024-09-10T22:35:04.062941+0000","last_became_active":"2024-09-10T22:35:02.059208+0000","last_became_peered":"2024-09-10T22:35:02.059208+0000","last_unstale":"2024-09-10T22:35:04.062941+0000","last_undegraded":"2024-09-10T22:35:04.062941+0000","last_fullsized":"2024-09-10T22:35:04.062941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:21.383738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063522+0000","last_change":"2024-09-10T22:35:03.067755+0000","last_active":"2024-09-10T22:35:04.063522+0000","last_peered":"2024-09-10T22:35:04.063522+0000","last_clean":"2024-09-10T22:35:04.063522+0000","last_became_active":"2024-09-10T22:35:03.067489+0000","last_became_peered":"2024-09-10T22:35:03.067489+0000","last_unstale":"2024-09-10T22:35:04.063522+0000","last_undegraded":"2024-09-10T22:35:04.063522+0000","last_fullsized":"2024-09-10T22:35:04.063522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:22.388769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060864+0000","last_change":"2024-09-10T22:35:02.063524+0000","last_active":"2024-09-10T22:35:04.060864+0000","last_peered":"2024-09-10T22:35:04.060864+0000","last_clean":"2024-09-10T22:35:04.060864+0000","last_became_active":"2024-09-10T22:35:02.063232+0000","last_became_peered":"2024-09-10T22:35:02.063232+0000","last_unstale":"2024-09-10T22:35:04.060864+0000","last_undegraded":"2024-09-10T22:35:04.060864+0000","last_fullsized":"2024-09-10T22:35:04.060864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062991+0000","last_change":"2024-09-10T22:35:03.071953+0000","last_active":"2024-09-10T22:35:04.062991+0000","last_peered":"2024-09-10T22:35:04.062991+0000","last_clean":"2024-09-10T22:35:04.062991+0000","last_became_active":"2024-09-10T22:35:03.071660+0000","last_became_peered":"2024-09-10T22:35:03.071660+0000","last_unstale":"2024-09-10T22:35:04.062991+0000","last_undegraded":"2024-09-10T22:35:04.062991+0000","last_fullsized":"2024-09-10T22:35:04.062991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:15.659782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284091+0000","last_change":"2024-09-10T22:35:02.067388+0000","last_active":"2024-09-10T22:35:04.284091+0000","last_peered":"2024-09-10T22:35:04.284091+0000","last_clean":"2024-09-10T22:35:04.284091+0000","last_became_active":"2024-09-10T22:35:02.067088+0000","last_became_peered":"2024-09-10T22:35:02.067088+0000","last_unstale":"2024-09-10T22:35:04.284091+0000","last_undegraded":"2024-09-10T22:35:04.284091+0000","last_fullsized":"2024-09-10T22:35:04.284091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:36.542782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060999+0000","last_change":"2024-09-10T22:35:03.058800+0000","last_active":"2024-09-10T22:35:04.060999+0000","last_peered":"2024-09-10T22:35:04.060999+0000","last_clean":"2024-09-10T22:35:04.060999+0000","last_became_active":"2024-09-10T22:35:03.058503+0000","last_became_peered":"2024-09-10T22:35:03.058503+0000","last_unstale":"2024-09-10T22:35:04.060999+0000","last_undegraded":"2024-09-10T22:35:04.060999+0000","last_fullsized":"2024-09-10T22:35:04.060999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.058112+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.057838+0000","last_became_peered":"2024-09-10T22:35:02.057838+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:20.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283672+0000","last_change":"2024-09-10T22:35:03.072054+0000","last_active":"2024-09-10T22:35:04.283672+0000","last_peered":"2024-09-10T22:35:04.283672+0000","last_clean":"2024-09-10T22:35:04.283672+0000","last_became_active":"2024-09-10T22:35:03.071739+0000","last_became_peered":"2024-09-10T22:35:03.071739+0000","last_unstale":"2024-09-10T22:35:04.283672+0000","last_undegraded":"2024-09-10T22:35:04.283672+0000","last_fullsized":"2024-09-10T22:35:04.283672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283676+0000","last_change":"2024-09-10T22:35:02.064151+0000","last_active":"2024-09-10T22:35:04.283676+0000","last_peered":"2024-09-10T22:35:04.283676+0000","last_clean":"2024-09-10T22:35:04.283676+0000","last_became_active":"2024-09-10T22:35:02.063573+0000","last_became_peered":"2024-09-10T22:35:02.063573+0000","last_unstale":"2024-09-10T22:35:04.283676+0000","last_undegraded":"2024-09-10T22:35:04.283676+0000","last_fullsized":"2024-09-10T22:35:04.283676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:21.244402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283654+0000","last_change":"2024-09-10T22:35:03.057069+0000","last_active":"2024-09-10T22:35:04.283654+0000","last_peered":"2024-09-10T22:35:04.283654+0000","last_clean":"2024-09-10T22:35:04.283654+0000","last_became_active":"2024-09-10T22:35:03.056760+0000","last_became_peered":"2024-09-10T22:35:03.056760+0000","last_unstale":"2024-09-10T22:35:04.283654+0000","last_undegraded":"2024-09-10T22:35:04.283654+0000","last_fullsized":"2024-09-10T22:35:04.283654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:11.516935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282652+0000","last_change":"2024-09-10T22:35:02.066275+0000","last_active":"2024-09-10T22:35:04.282652+0000","last_peered":"2024-09-10T22:35:04.282652+0000","last_clean":"2024-09-10T22:35:04.282652+0000","last_became_active":"2024-09-10T22:35:02.065887+0000","last_became_peered":"2024-09-10T22:35:02.065887+0000","last_unstale":"2024-09-10T22:35:04.282652+0000","last_undegraded":"2024-09-10T22:35:04.282652+0000","last_fullsized":"2024-09-10T22:35:04.282652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:09.305553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062290+0000","last_change":"2024-09-10T22:35:03.054840+0000","last_active":"2024-09-10T22:35:04.062290+0000","last_peered":"2024-09-10T22:35:04.062290+0000","last_clean":"2024-09-10T22:35:04.062290+0000","last_became_active":"2024-09-10T22:35:03.054562+0000","last_became_peered":"2024-09-10T22:35:03.054562+0000","last_unstale":"2024-09-10T22:35:04.062290+0000","last_undegraded":"2024-09-10T22:35:04.062290+0000","last_fullsized":"2024-09-10T22:35:04.062290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:29.675776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060764+0000","last_change":"2024-09-10T22:35:02.061273+0000","last_active":"2024-09-10T22:35:04.060764+0000","last_peered":"2024-09-10T22:35:04.060764+0000","last_clean":"2024-09-10T22:35:04.060764+0000","last_became_active":"2024-09-10T22:35:02.060953+0000","last_became_peered":"2024-09-10T22:35:02.060953+0000","last_unstale":"2024-09-10T22:35:04.060764+0000","last_undegraded":"2024-09-10T22:35:04.060764+0000","last_fullsized":"2024-09-10T22:35:04.060764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:32.172943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284309+0000","last_change":"2024-09-10T22:35:03.078424+0000","last_active":"2024-09-10T22:35:04.284309+0000","last_peered":"2024-09-10T22:35:04.284309+0000","last_clean":"2024-09-10T22:35:04.284309+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.284309+0000","last_undegraded":"2024-09-10T22:35:04.284309+0000","last_fullsized":"2024-09-10T22:35:04.284309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:48.985804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.061749+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.061248+0000","last_became_peered":"2024-09-10T22:35:02.061248+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:49.960770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283758+0000","last_change":"2024-09-10T22:35:03.076736+0000","last_active":"2024-09-10T22:35:04.283758+0000","last_peered":"2024-09-10T22:35:04.283758+0000","last_clean":"2024-09-10T22:35:04.283758+0000","last_became_active":"2024-09-10T22:35:03.075379+0000","last_became_peered":"2024-09-10T22:35:03.075379+0000","last_unstale":"2024-09-10T22:35:04.283758+0000","last_undegraded":"2024-09-10T22:35:04.283758+0000","last_fullsized":"2024-09-10T22:35:04.283758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.399411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284154+0000","last_change":"2024-09-10T22:35:02.065909+0000","last_active":"2024-09-10T22:35:04.284154+0000","last_peered":"2024-09-10T22:35:04.284154+0000","last_clean":"2024-09-10T22:35:04.284154+0000","last_became_active":"2024-09-10T22:35:02.065539+0000","last_became_peered":"2024-09-10T22:35:02.065539+0000","last_unstale":"2024-09-10T22:35:04.284154+0000","last_undegraded":"2024-09-10T22:35:04.284154+0000","last_fullsized":"2024-09-10T22:35:04.284154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:32.374399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283186+0000","last_change":"2024-09-10T22:35:03.060477+0000","last_active":"2024-09-10T22:35:04.283186+0000","last_peered":"2024-09-10T22:35:04.283186+0000","last_clean":"2024-09-10T22:35:04.283186+0000","last_became_active":"2024-09-10T22:35:03.060190+0000","last_became_peered":"2024-09-10T22:35:03.060190+0000","last_unstale":"2024-09-10T22:35:04.283186+0000","last_undegraded":"2024-09-10T22:35:04.283186+0000","last_fullsized":"2024-09-10T22:35:04.283186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:56.641563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062898+0000","last_change":"2024-09-10T22:35:03.060225+0000","last_active":"2024-09-10T22:35:04.062898+0000","last_peered":"2024-09-10T22:35:04.062898+0000","last_clean":"2024-09-10T22:35:04.062898+0000","last_became_active":"2024-09-10T22:35:03.059807+0000","last_became_peered":"2024-09-10T22:35:03.059807+0000","last_unstale":"2024-09-10T22:35:04.062898+0000","last_undegraded":"2024-09-10T22:35:04.062898+0000","last_fullsized":"2024-09-10T22:35:04.062898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:12.132002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283516+0000","last_change":"2024-09-10T22:35:02.066541+0000","last_active":"2024-09-10T22:35:04.283516+0000","last_peered":"2024-09-10T22:35:04.283516+0000","last_clean":"2024-09-10T22:35:04.283516+0000","last_became_active":"2024-09-10T22:35:02.066259+0000","last_became_peered":"2024-09-10T22:35:02.066259+0000","last_unstale":"2024-09-10T22:35:04.283516+0000","last_undegraded":"2024-09-10T22:35:04.283516+0000","last_fullsized":"2024-09-10T22:35:04.283516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:58.279821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062332+0000","last_change":"2024-09-10T22:35:03.068676+0000","last_active":"2024-09-10T22:35:04.062332+0000","last_peered":"2024-09-10T22:35:04.062332+0000","last_clean":"2024-09-10T22:35:04.062332+0000","last_became_active":"2024-09-10T22:35:03.068384+0000","last_became_peered":"2024-09-10T22:35:03.068384+0000","last_unstale":"2024-09-10T22:35:04.062332+0000","last_undegraded":"2024-09-10T22:35:04.062332+0000","last_fullsized":"2024-09-10T22:35:04.062332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:56.275057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283838+0000","last_change":"2024-09-10T22:35:02.065459+0000","last_active":"2024-09-10T22:35:04.283838+0000","last_peered":"2024-09-10T22:35:04.283838+0000","last_clean":"2024-09-10T22:35:04.283838+0000","last_became_active":"2024-09-10T22:35:02.065145+0000","last_became_peered":"2024-09-10T22:35:02.065145+0000","last_unstale":"2024-09-10T22:35:04.283838+0000","last_undegraded":"2024-09-10T22:35:04.283838+0000","last_fullsized":"2024-09-10T22:35:04.283838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:21.383738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283725+0000","last_change":"2024-09-10T22:35:03.063955+0000","last_active":"2024-09-10T22:35:04.283725+0000","last_peered":"2024-09-10T22:35:04.283725+0000","last_clean":"2024-09-10T22:35:04.283725+0000","last_became_active":"2024-09-10T22:35:03.063679+0000","last_became_peered":"2024-09-10T22:35:03.063679+0000","last_unstale":"2024-09-10T22:35:04.283725+0000","last_undegraded":"2024-09-10T22:35:04.283725+0000","last_fullsized":"2024-09-10T22:35:04.283725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:56.641563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283509+0000","last_change":"2024-09-10T22:35:02.062827+0000","last_active":"2024-09-10T22:35:04.283509+0000","last_peered":"2024-09-10T22:35:04.283509+0000","last_clean":"2024-09-10T22:35:04.283509+0000","last_became_active":"2024-09-10T22:35:02.062342+0000","last_became_peered":"2024-09-10T22:35:02.062342+0000","last_unstale":"2024-09-10T22:35:04.283509+0000","last_undegraded":"2024-09-10T22:35:04.283509+0000","last_fullsized":"2024-09-10T22:35:04.283509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283072+0000","last_change":"2024-09-10T22:35:03.080815+0000","last_active":"2024-09-10T22:35:04.283072+0000","last_peered":"2024-09-10T22:35:04.283072+0000","last_clean":"2024-09-10T22:35:04.283072+0000","last_became_active":"2024-09-10T22:35:03.080484+0000","last_became_peered":"2024-09-10T22:35:03.080484+0000","last_unstale":"2024-09-10T22:35:04.283072+0000","last_undegraded":"2024-09-10T22:35:04.283072+0000","last_fullsized":"2024-09-10T22:35:04.283072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:58.700652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283792+0000","last_change":"2024-09-10T22:35:02.065788+0000","last_active":"2024-09-10T22:35:04.283792+0000","last_peered":"2024-09-10T22:35:04.283792+0000","last_clean":"2024-09-10T22:35:04.283792+0000","last_became_active":"2024-09-10T22:35:02.065500+0000","last_became_peered":"2024-09-10T22:35:02.065500+0000","last_unstale":"2024-09-10T22:35:04.283792+0000","last_undegraded":"2024-09-10T22:35:04.283792+0000","last_fullsized":"2024-09-10T22:35:04.283792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:10.573361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":663271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":72411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":72447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":815104,"data_stored":663327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:16.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:16.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.396+0000 7f9b849ad640 1 -- 172.21.15.66:0/2228689921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 msgr2=0x7f9b800d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.397+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2228689921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 0x7f9b800d1cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009bb0 tx=0x7f9b7802a7a0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.397+0000 7f9b849ad640 1 -- 172.21.15.66:0/2228689921 shutdown_connections 2024-09-10T22:35:16.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.397+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2228689921 >> v2:172.21.15.94:3300/0 conn(0x7f9b800b3380 0x7f9b800d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.397+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2228689921 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 0x7f9b800d1cb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.397+0000 7f9b849ad640 1 -- 172.21.15.66:0/2228689921 >> 172.21.15.66:0/2228689921 conn(0x7f9b8009aa70 msgr2=0x7f9b8009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:16.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.397+0000 7f9b849ad640 1 -- 172.21.15.66:0/2228689921 shutdown_connections 2024-09-10T22:35:16.399 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.397+0000 7f9b849ad640 1 -- 172.21.15.66:0/2228689921 wait complete. 2024-09-10T22:35:16.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.398+0000 7f9b849ad640 1 Processor -- start 2024-09-10T22:35:16.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.398+0000 7f9b849ad640 1 -- start start 2024-09-10T22:35:16.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b849ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b800afe30 0x7f9b8015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b849ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b800b3380 0x7f9b801f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b849ad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 0x7f9b801f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b849ad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b800b09f0 con 0x7f9b800b3380 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b849ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b800b2f50 con 0x7f9b800afe30 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b849ad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9b800b0b70 con 0x7f9b8015bf00 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b800b3380 0x7f9b801f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b800b3380 0x7f9b801f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35926/0 (socket says 172.21.15.66:35926) 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 -- 172.21.15.66:0/2868853002 learned_addr learned my addr 172.21.15.66:0/2868853002 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7e59c640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b800afe30 0x7f9b8015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7ed9d640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 0x7f9b801f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 -- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 msgr2=0x7f9b801f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 0x7f9b801f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 -- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b800afe30 msgr2=0x7f9b8015c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b800afe30 0x7f9b8015c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 -- 172.21.15.66:0/2868853002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8006dc30 con 0x7f9b800b3380 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7e59c640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b800afe30 0x7f9b8015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7ed9d640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 0x7f9b801f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.399+0000 7f9b7dd9b640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b800b3380 0x7f9b801f48b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800e9f0 tx=0x7f9b6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.400+0000 7f9b76ffd640 1 -- 172.21.15.66:0/2868853002 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800cd80 con 0x7f9b800b3380 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.400+0000 7f9b76ffd640 1 -- 172.21.15.66:0/2868853002 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b68004020 con 0x7f9b800b3380 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.400+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b801f5190 con 0x7f9b800b3380 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.400+0000 7f9b76ffd640 1 -- 172.21.15.66:0/2868853002 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68004430 con 0x7f9b800b3380 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.400+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8015f370 con 0x7f9b800b3380 2024-09-10T22:35:16.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.401+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b800b1060 con 0x7f9b800b3380 2024-09-10T22:35:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.401+0000 7f9b76ffd640 1 -- 172.21.15.66:0/2868853002 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9b680035f0 con 0x7f9b800b3380 2024-09-10T22:35:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.402+0000 7f9b76ffd640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b60077f70 0x7f9b6007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.402+0000 7f9b76ffd640 1 -- 172.21.15.66:0/2868853002 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9b6809a7a0 con 0x7f9b800b3380 2024-09-10T22:35:16.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.402+0000 7f9b7e59c640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b60077f70 0x7f9b6007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.402+0000 7f9b7e59c640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b60077f70 0x7f9b6007a430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b800b1c50 tx=0x7f9b78004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:16.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.406+0000 7f9b76ffd640 1 -- 172.21.15.66:0/2868853002 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9b680633f0 con 0x7f9b800b3380 2024-09-10T22:35:16.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.586+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b800b66a0 con 0x7f9b60077f70 2024-09-10T22:35:16.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.591+0000 7f9b76ffd640 1 -- 172.21.15.66:0/2868853002 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f9b800b66a0 con 0x7f9b60077f70 2024-09-10T22:35:16.594 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:16.595 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b60077f70 msgr2=0x7f9b6007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b60077f70 0x7f9b6007a430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b800b1c50 tx=0x7f9b78004ab0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b800b3380 msgr2=0x7f9b801f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b800b3380 0x7f9b801f48b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800e9f0 tx=0x7f9b6800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 shutdown_connections 2024-09-10T22:35:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b60077f70 0x7f9b6007a430 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b8015bf00 0x7f9b801f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b800b3380 0x7f9b801f48b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 --2- 172.21.15.66:0/2868853002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b800afe30 0x7f9b8015c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 >> 172.21.15.66:0/2868853002 conn(0x7f9b8009aa70 msgr2=0x7f9b80071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:16.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 shutdown_connections 2024-09-10T22:35:16.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.594+0000 7f9b849ad640 1 -- 172.21.15.66:0/2868853002 wait complete. 2024-09-10T22:35:16.606 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T22:35:14.595453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235760,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738960,"statfs":{"total":773094113280,"available":772852695040,"internally_reserved":0,"allocated":2924544,"data_stored":1764124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.535525"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283978+0000","last_change":"2024-09-10T22:35:03.064865+0000","last_active":"2024-09-10T22:35:04.283978+0000","last_peered":"2024-09-10T22:35:04.283978+0000","last_clean":"2024-09-10T22:35:04.283978+0000","last_became_active":"2024-09-10T22:35:03.064552+0000","last_became_peered":"2024-09-10T22:35:03.064552+0000","last_unstale":"2024-09-10T22:35:04.283978+0000","last_undegraded":"2024-09-10T22:35:04.283978+0000","last_fullsized":"2024-09-10T22:35:04.283978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:42.990367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060843+0000","last_change":"2024-09-10T22:35:03.065907+0000","last_active":"2024-09-10T22:35:04.060843+0000","last_peered":"2024-09-10T22:35:04.060843+0000","last_clean":"2024-09-10T22:35:04.060843+0000","last_became_active":"2024-09-10T22:35:03.065607+0000","last_became_peered":"2024-09-10T22:35:03.065607+0000","last_unstale":"2024-09-10T22:35:04.060843+0000","last_undegraded":"2024-09-10T22:35:04.060843+0000","last_fullsized":"2024-09-10T22:35:04.060843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283605+0000","last_change":"2024-09-10T22:35:03.063096+0000","last_active":"2024-09-10T22:35:04.283605+0000","last_peered":"2024-09-10T22:35:04.283605+0000","last_clean":"2024-09-10T22:35:04.283605+0000","last_became_active":"2024-09-10T22:35:03.057986+0000","last_became_peered":"2024-09-10T22:35:03.057986+0000","last_unstale":"2024-09-10T22:35:04.283605+0000","last_undegraded":"2024-09-10T22:35:04.283605+0000","last_fullsized":"2024-09-10T22:35:04.283605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:41.190200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062415+0000","last_change":"2024-09-10T22:35:03.076412+0000","last_active":"2024-09-10T22:35:04.062415+0000","last_peered":"2024-09-10T22:35:04.062415+0000","last_clean":"2024-09-10T22:35:04.062415+0000","last_became_active":"2024-09-10T22:35:03.076037+0000","last_became_peered":"2024-09-10T22:35:03.076037+0000","last_unstale":"2024-09-10T22:35:04.062415+0000","last_undegraded":"2024-09-10T22:35:04.062415+0000","last_fullsized":"2024-09-10T22:35:04.062415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:35.334336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284073+0000","last_change":"2024-09-10T22:35:03.061394+0000","last_active":"2024-09-10T22:35:04.284073+0000","last_peered":"2024-09-10T22:35:04.284073+0000","last_clean":"2024-09-10T22:35:04.284073+0000","last_became_active":"2024-09-10T22:35:03.061111+0000","last_became_peered":"2024-09-10T22:35:03.061111+0000","last_unstale":"2024-09-10T22:35:04.284073+0000","last_undegraded":"2024-09-10T22:35:04.284073+0000","last_fullsized":"2024-09-10T22:35:04.284073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:48.372055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062681+0000","last_change":"2024-09-10T22:35:03.062367+0000","last_active":"2024-09-10T22:35:04.062681+0000","last_peered":"2024-09-10T22:35:04.062681+0000","last_clean":"2024-09-10T22:35:04.062681+0000","last_became_active":"2024-09-10T22:35:03.061999+0000","last_became_peered":"2024-09-10T22:35:03.061999+0000","last_unstale":"2024-09-10T22:35:04.062681+0000","last_undegraded":"2024-09-10T22:35:04.062681+0000","last_fullsized":"2024-09-10T22:35:04.062681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:42.990367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282913+0000","last_change":"2024-09-10T22:35:03.065678+0000","last_active":"2024-09-10T22:35:04.282913+0000","last_peered":"2024-09-10T22:35:04.282913+0000","last_clean":"2024-09-10T22:35:04.282913+0000","last_became_active":"2024-09-10T22:35:03.065414+0000","last_became_peered":"2024-09-10T22:35:03.065414+0000","last_unstale":"2024-09-10T22:35:04.282913+0000","last_undegraded":"2024-09-10T22:35:04.282913+0000","last_fullsized":"2024-09-10T22:35:04.282913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:56.275057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283932+0000","last_change":"2024-09-10T22:35:03.072149+0000","last_active":"2024-09-10T22:35:04.283932+0000","last_peered":"2024-09-10T22:35:04.283932+0000","last_clean":"2024-09-10T22:35:04.283932+0000","last_became_active":"2024-09-10T22:35:03.071819+0000","last_became_peered":"2024-09-10T22:35:03.071819+0000","last_unstale":"2024-09-10T22:35:04.283932+0000","last_undegraded":"2024-09-10T22:35:04.283932+0000","last_fullsized":"2024-09-10T22:35:04.283932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:26.466821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284074+0000","last_change":"2024-09-10T22:35:03.068359+0000","last_active":"2024-09-10T22:35:04.284074+0000","last_peered":"2024-09-10T22:35:04.284074+0000","last_clean":"2024-09-10T22:35:04.284074+0000","last_became_active":"2024-09-10T22:35:03.067889+0000","last_became_peered":"2024-09-10T22:35:03.067889+0000","last_unstale":"2024-09-10T22:35:04.284074+0000","last_undegraded":"2024-09-10T22:35:04.284074+0000","last_fullsized":"2024-09-10T22:35:04.284074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:06.735630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283225+0000","last_change":"2024-09-10T22:35:03.061509+0000","last_active":"2024-09-10T22:35:04.283225+0000","last_peered":"2024-09-10T22:35:04.283225+0000","last_clean":"2024-09-10T22:35:04.283225+0000","last_became_active":"2024-09-10T22:35:03.061170+0000","last_became_peered":"2024-09-10T22:35:03.061170+0000","last_unstale":"2024-09-10T22:35:04.283225+0000","last_undegraded":"2024-09-10T22:35:04.283225+0000","last_fullsized":"2024-09-10T22:35:04.283225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:26.993716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283408+0000","last_change":"2024-09-10T22:35:03.061387+0000","last_active":"2024-09-10T22:35:04.283408+0000","last_peered":"2024-09-10T22:35:04.283408+0000","last_clean":"2024-09-10T22:35:04.283408+0000","last_became_active":"2024-09-10T22:35:03.061088+0000","last_became_peered":"2024-09-10T22:35:03.061088+0000","last_unstale":"2024-09-10T22:35:04.283408+0000","last_undegraded":"2024-09-10T22:35:04.283408+0000","last_fullsized":"2024-09-10T22:35:04.283408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:31.398073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063686+0000","last_change":"2024-09-10T22:35:03.058702+0000","last_active":"2024-09-10T22:35:04.063686+0000","last_peered":"2024-09-10T22:35:04.063686+0000","last_clean":"2024-09-10T22:35:04.063686+0000","last_became_active":"2024-09-10T22:35:03.058419+0000","last_became_peered":"2024-09-10T22:35:03.058419+0000","last_unstale":"2024-09-10T22:35:04.063686+0000","last_undegraded":"2024-09-10T22:35:04.063686+0000","last_fullsized":"2024-09-10T22:35:04.063686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:59.284853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283542+0000","last_change":"2024-09-10T22:35:03.080808+0000","last_active":"2024-09-10T22:35:04.283542+0000","last_peered":"2024-09-10T22:35:04.283542+0000","last_clean":"2024-09-10T22:35:04.283542+0000","last_became_active":"2024-09-10T22:35:03.080467+0000","last_became_peered":"2024-09-10T22:35:03.080467+0000","last_unstale":"2024-09-10T22:35:04.283542+0000","last_undegraded":"2024-09-10T22:35:04.283542+0000","last_fullsized":"2024-09-10T22:35:04.283542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:56.162264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282815+0000","last_change":"2024-09-10T22:35:03.057789+0000","last_active":"2024-09-10T22:35:04.282815+0000","last_peered":"2024-09-10T22:35:04.282815+0000","last_clean":"2024-09-10T22:35:04.282815+0000","last_became_active":"2024-09-10T22:35:03.057450+0000","last_became_peered":"2024-09-10T22:35:03.057450+0000","last_unstale":"2024-09-10T22:35:04.282815+0000","last_undegraded":"2024-09-10T22:35:04.282815+0000","last_fullsized":"2024-09-10T22:35:04.282815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:30.343040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060665+0000","last_change":"2024-09-10T22:35:03.062056+0000","last_active":"2024-09-10T22:35:04.060665+0000","last_peered":"2024-09-10T22:35:04.060665+0000","last_clean":"2024-09-10T22:35:04.060665+0000","last_became_active":"2024-09-10T22:35:03.061769+0000","last_became_peered":"2024-09-10T22:35:03.061769+0000","last_unstale":"2024-09-10T22:35:04.060665+0000","last_undegraded":"2024-09-10T22:35:04.060665+0000","last_fullsized":"2024-09-10T22:35:04.060665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:02.406217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.061953+0000","last_change":"2024-09-10T22:35:03.067251+0000","last_active":"2024-09-10T22:35:04.061953+0000","last_peered":"2024-09-10T22:35:04.061953+0000","last_clean":"2024-09-10T22:35:04.061953+0000","last_became_active":"2024-09-10T22:35:03.066980+0000","last_became_peered":"2024-09-10T22:35:03.066980+0000","last_unstale":"2024-09-10T22:35:04.061953+0000","last_undegraded":"2024-09-10T22:35:04.061953+0000","last_fullsized":"2024-09-10T22:35:04.061953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:45.349941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062881+0000","last_change":"2024-09-10T22:35:03.070627+0000","last_active":"2024-09-10T22:35:04.062881+0000","last_peered":"2024-09-10T22:35:04.062881+0000","last_clean":"2024-09-10T22:35:04.062881+0000","last_became_active":"2024-09-10T22:35:03.070365+0000","last_became_peered":"2024-09-10T22:35:03.070365+0000","last_unstale":"2024-09-10T22:35:04.062881+0000","last_undegraded":"2024-09-10T22:35:04.062881+0000","last_fullsized":"2024-09-10T22:35:04.062881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:56.758850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282700+0000","last_change":"2024-09-10T22:35:03.060118+0000","last_active":"2024-09-10T22:35:04.282700+0000","last_peered":"2024-09-10T22:35:04.282700+0000","last_clean":"2024-09-10T22:35:04.282700+0000","last_became_active":"2024-09-10T22:35:03.059799+0000","last_became_peered":"2024-09-10T22:35:03.059799+0000","last_unstale":"2024-09-10T22:35:04.282700+0000","last_undegraded":"2024-09-10T22:35:04.282700+0000","last_fullsized":"2024-09-10T22:35:04.282700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:50.965802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284045+0000","last_change":"2024-09-10T22:35:03.062861+0000","last_active":"2024-09-10T22:35:04.284045+0000","last_peered":"2024-09-10T22:35:04.284045+0000","last_clean":"2024-09-10T22:35:04.284045+0000","last_became_active":"2024-09-10T22:35:03.062586+0000","last_became_peered":"2024-09-10T22:35:03.062586+0000","last_unstale":"2024-09-10T22:35:04.284045+0000","last_undegraded":"2024-09-10T22:35:04.284045+0000","last_fullsized":"2024-09-10T22:35:04.284045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:22.142634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283055+0000","last_change":"2024-09-10T22:35:03.070244+0000","last_active":"2024-09-10T22:35:04.283055+0000","last_peered":"2024-09-10T22:35:04.283055+0000","last_clean":"2024-09-10T22:35:04.283055+0000","last_became_active":"2024-09-10T22:35:03.067927+0000","last_became_peered":"2024-09-10T22:35:03.067927+0000","last_unstale":"2024-09-10T22:35:04.283055+0000","last_undegraded":"2024-09-10T22:35:04.283055+0000","last_fullsized":"2024-09-10T22:35:04.283055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:51.838173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283820+0000","last_change":"2024-09-10T22:35:03.065175+0000","last_active":"2024-09-10T22:35:04.283820+0000","last_peered":"2024-09-10T22:35:04.283820+0000","last_clean":"2024-09-10T22:35:04.283820+0000","last_became_active":"2024-09-10T22:35:03.064880+0000","last_became_peered":"2024-09-10T22:35:03.064880+0000","last_unstale":"2024-09-10T22:35:04.283820+0000","last_undegraded":"2024-09-10T22:35:04.283820+0000","last_fullsized":"2024-09-10T22:35:04.283820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:34.515907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062375+0000","last_change":"2024-09-10T22:35:03.069577+0000","last_active":"2024-09-10T22:35:04.062375+0000","last_peered":"2024-09-10T22:35:04.062375+0000","last_clean":"2024-09-10T22:35:04.062375+0000","last_became_active":"2024-09-10T22:35:03.069224+0000","last_became_peered":"2024-09-10T22:35:03.069224+0000","last_unstale":"2024-09-10T22:35:04.062375+0000","last_undegraded":"2024-09-10T22:35:04.062375+0000","last_fullsized":"2024-09-10T22:35:04.062375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:16.644778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283468+0000","last_change":"2024-09-10T22:35:03.076617+0000","last_active":"2024-09-10T22:35:04.283468+0000","last_peered":"2024-09-10T22:35:04.283468+0000","last_clean":"2024-09-10T22:35:04.283468+0000","last_became_active":"2024-09-10T22:35:03.075527+0000","last_became_peered":"2024-09-10T22:35:03.075527+0000","last_unstale":"2024-09-10T22:35:04.283468+0000","last_undegraded":"2024-09-10T22:35:04.283468+0000","last_fullsized":"2024-09-10T22:35:04.283468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062216+0000","last_change":"2024-09-10T22:35:03.072643+0000","last_active":"2024-09-10T22:35:04.062216+0000","last_peered":"2024-09-10T22:35:04.062216+0000","last_clean":"2024-09-10T22:35:04.062216+0000","last_became_active":"2024-09-10T22:35:03.072354+0000","last_became_peered":"2024-09-10T22:35:03.072354+0000","last_unstale":"2024-09-10T22:35:04.062216+0000","last_undegraded":"2024-09-10T22:35:04.062216+0000","last_fullsized":"2024-09-10T22:35:04.062216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:25.832743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283390+0000","last_change":"2024-09-10T22:35:03.076662+0000","last_active":"2024-09-10T22:35:04.283390+0000","last_peered":"2024-09-10T22:35:04.283390+0000","last_clean":"2024-09-10T22:35:04.283390+0000","last_became_active":"2024-09-10T22:35:03.075696+0000","last_became_peered":"2024-09-10T22:35:03.075696+0000","last_unstale":"2024-09-10T22:35:04.283390+0000","last_undegraded":"2024-09-10T22:35:04.283390+0000","last_fullsized":"2024-09-10T22:35:04.283390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:59.070897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283416+0000","last_change":"2024-09-10T22:35:03.070886+0000","last_active":"2024-09-10T22:35:04.283416+0000","last_peered":"2024-09-10T22:35:04.283416+0000","last_clean":"2024-09-10T22:35:04.283416+0000","last_became_active":"2024-09-10T22:35:03.070569+0000","last_became_peered":"2024-09-10T22:35:03.070569+0000","last_unstale":"2024-09-10T22:35:04.283416+0000","last_undegraded":"2024-09-10T22:35:04.283416+0000","last_fullsized":"2024-09-10T22:35:04.283416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:25.138442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060768+0000","last_change":"2024-09-10T22:35:03.074633+0000","last_active":"2024-09-10T22:35:04.060768+0000","last_peered":"2024-09-10T22:35:04.060768+0000","last_clean":"2024-09-10T22:35:04.060768+0000","last_became_active":"2024-09-10T22:35:03.074332+0000","last_became_peered":"2024-09-10T22:35:03.074332+0000","last_unstale":"2024-09-10T22:35:04.060768+0000","last_undegraded":"2024-09-10T22:35:04.060768+0000","last_fullsized":"2024-09-10T22:35:04.060768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:07.079604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283311+0000","last_change":"2024-09-10T22:35:03.077344+0000","last_active":"2024-09-10T22:35:04.283311+0000","last_peered":"2024-09-10T22:35:04.283311+0000","last_clean":"2024-09-10T22:35:04.283311+0000","last_became_active":"2024-09-10T22:35:03.075668+0000","last_became_peered":"2024-09-10T22:35:03.075668+0000","last_unstale":"2024-09-10T22:35:04.283311+0000","last_undegraded":"2024-09-10T22:35:04.283311+0000","last_fullsized":"2024-09-10T22:35:04.283311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283878+0000","last_change":"2024-09-10T22:35:03.078565+0000","last_active":"2024-09-10T22:35:04.283878+0000","last_peered":"2024-09-10T22:35:04.283878+0000","last_clean":"2024-09-10T22:35:04.283878+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.283878+0000","last_undegraded":"2024-09-10T22:35:04.283878+0000","last_fullsized":"2024-09-10T22:35:04.283878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:57.500600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062623+0000","last_change":"2024-09-10T22:35:03.075919+0000","last_active":"2024-09-10T22:35:04.062623+0000","last_peered":"2024-09-10T22:35:04.062623+0000","last_clean":"2024-09-10T22:35:04.062623+0000","last_became_active":"2024-09-10T22:35:03.075584+0000","last_became_peered":"2024-09-10T22:35:03.075584+0000","last_unstale":"2024-09-10T22:35:04.062623+0000","last_undegraded":"2024-09-10T22:35:04.062623+0000","last_fullsized":"2024-09-10T22:35:04.062623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:57.797898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282910+0000","last_change":"2024-09-10T22:35:03.058708+0000","last_active":"2024-09-10T22:35:04.282910+0000","last_peered":"2024-09-10T22:35:04.282910+0000","last_clean":"2024-09-10T22:35:04.282910+0000","last_became_active":"2024-09-10T22:35:03.058399+0000","last_became_peered":"2024-09-10T22:35:03.058399+0000","last_unstale":"2024-09-10T22:35:04.282910+0000","last_undegraded":"2024-09-10T22:35:04.282910+0000","last_fullsized":"2024-09-10T22:35:04.282910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:21.971330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282922+0000","last_change":"2024-09-10T22:35:03.066152+0000","last_active":"2024-09-10T22:35:04.282922+0000","last_peered":"2024-09-10T22:35:04.282922+0000","last_clean":"2024-09-10T22:35:04.282922+0000","last_became_active":"2024-09-10T22:35:03.065897+0000","last_became_peered":"2024-09-10T22:35:03.065897+0000","last_unstale":"2024-09-10T22:35:04.282922+0000","last_undegraded":"2024-09-10T22:35:04.282922+0000","last_fullsized":"2024-09-10T22:35:04.282922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:01.896261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063193+0000","last_change":"2024-09-10T22:35:03.062529+0000","last_active":"2024-09-10T22:35:04.063193+0000","last_peered":"2024-09-10T22:35:04.063193+0000","last_clean":"2024-09-10T22:35:04.063193+0000","last_became_active":"2024-09-10T22:35:03.059720+0000","last_became_peered":"2024-09-10T22:35:03.059720+0000","last_unstale":"2024-09-10T22:35:04.063193+0000","last_undegraded":"2024-09-10T22:35:04.063193+0000","last_fullsized":"2024-09-10T22:35:04.063193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:45.451161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283873+0000","last_change":"2024-09-10T22:35:02.060438+0000","last_active":"2024-09-10T22:35:04.283873+0000","last_peered":"2024-09-10T22:35:04.283873+0000","last_clean":"2024-09-10T22:35:04.283873+0000","last_became_active":"2024-09-10T22:35:02.060124+0000","last_became_peered":"2024-09-10T22:35:02.060124+0000","last_unstale":"2024-09-10T22:35:04.283873+0000","last_undegraded":"2024-09-10T22:35:04.283873+0000","last_fullsized":"2024-09-10T22:35:04.283873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283981+0000","last_change":"2024-09-10T22:35:03.068471+0000","last_active":"2024-09-10T22:35:04.283981+0000","last_peered":"2024-09-10T22:35:04.283981+0000","last_clean":"2024-09-10T22:35:04.283981+0000","last_became_active":"2024-09-10T22:35:03.068079+0000","last_became_peered":"2024-09-10T22:35:03.068079+0000","last_unstale":"2024-09-10T22:35:04.283981+0000","last_undegraded":"2024-09-10T22:35:04.283981+0000","last_fullsized":"2024-09-10T22:35:04.283981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:53.676311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283156+0000","last_change":"2024-09-10T22:35:02.060297+0000","last_active":"2024-09-10T22:35:04.283156+0000","last_peered":"2024-09-10T22:35:04.283156+0000","last_clean":"2024-09-10T22:35:04.283156+0000","last_became_active":"2024-09-10T22:35:02.059966+0000","last_became_peered":"2024-09-10T22:35:02.059966+0000","last_unstale":"2024-09-10T22:35:04.283156+0000","last_undegraded":"2024-09-10T22:35:04.283156+0000","last_fullsized":"2024-09-10T22:35:04.283156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.073637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283633+0000","last_change":"2024-09-10T22:35:02.058735+0000","last_active":"2024-09-10T22:35:04.283633+0000","last_peered":"2024-09-10T22:35:04.283633+0000","last_clean":"2024-09-10T22:35:04.283633+0000","last_became_active":"2024-09-10T22:35:02.058425+0000","last_became_peered":"2024-09-10T22:35:02.058425+0000","last_unstale":"2024-09-10T22:35:04.283633+0000","last_undegraded":"2024-09-10T22:35:04.283633+0000","last_fullsized":"2024-09-10T22:35:04.283633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284127+0000","last_change":"2024-09-10T22:35:03.070043+0000","last_active":"2024-09-10T22:35:04.284127+0000","last_peered":"2024-09-10T22:35:04.284127+0000","last_clean":"2024-09-10T22:35:04.284127+0000","last_became_active":"2024-09-10T22:35:03.069418+0000","last_became_peered":"2024-09-10T22:35:03.069418+0000","last_unstale":"2024-09-10T22:35:04.284127+0000","last_undegraded":"2024-09-10T22:35:04.284127+0000","last_fullsized":"2024-09-10T22:35:04.284127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:23.917793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062027+0000","last_change":"2024-09-10T22:35:02.056871+0000","last_active":"2024-09-10T22:35:04.062027+0000","last_peered":"2024-09-10T22:35:04.062027+0000","last_clean":"2024-09-10T22:35:04.062027+0000","last_became_active":"2024-09-10T22:35:02.056541+0000","last_became_peered":"2024-09-10T22:35:02.056541+0000","last_unstale":"2024-09-10T22:35:04.062027+0000","last_undegraded":"2024-09-10T22:35:04.062027+0000","last_fullsized":"2024-09-10T22:35:04.062027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:58.279821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060647+0000","last_change":"2024-09-10T22:35:03.074156+0000","last_active":"2024-09-10T22:35:04.060647+0000","last_peered":"2024-09-10T22:35:04.060647+0000","last_clean":"2024-09-10T22:35:04.060647+0000","last_became_active":"2024-09-10T22:35:03.073836+0000","last_became_peered":"2024-09-10T22:35:03.073836+0000","last_unstale":"2024-09-10T22:35:04.060647+0000","last_undegraded":"2024-09-10T22:35:04.060647+0000","last_fullsized":"2024-09-10T22:35:04.060647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283581+0000","last_change":"2024-09-10T22:35:02.063854+0000","last_active":"2024-09-10T22:35:04.283581+0000","last_peered":"2024-09-10T22:35:04.283581+0000","last_clean":"2024-09-10T22:35:04.283581+0000","last_became_active":"2024-09-10T22:35:02.063214+0000","last_became_peered":"2024-09-10T22:35:02.063214+0000","last_unstale":"2024-09-10T22:35:04.283581+0000","last_undegraded":"2024-09-10T22:35:04.283581+0000","last_fullsized":"2024-09-10T22:35:04.283581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:23.311121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062461+0000","last_change":"2024-09-10T22:35:03.069518+0000","last_active":"2024-09-10T22:35:04.062461+0000","last_peered":"2024-09-10T22:35:04.062461+0000","last_clean":"2024-09-10T22:35:04.062461+0000","last_became_active":"2024-09-10T22:35:03.069271+0000","last_became_peered":"2024-09-10T22:35:03.069271+0000","last_unstale":"2024-09-10T22:35:04.062461+0000","last_undegraded":"2024-09-10T22:35:04.062461+0000","last_fullsized":"2024-09-10T22:35:04.062461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:28.595416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062604+0000","last_change":"2024-09-10T22:35:02.060887+0000","last_active":"2024-09-10T22:35:04.062604+0000","last_peered":"2024-09-10T22:35:04.062604+0000","last_clean":"2024-09-10T22:35:04.062604+0000","last_became_active":"2024-09-10T22:35:02.060054+0000","last_became_peered":"2024-09-10T22:35:02.060054+0000","last_unstale":"2024-09-10T22:35:04.062604+0000","last_undegraded":"2024-09-10T22:35:04.062604+0000","last_fullsized":"2024-09-10T22:35:04.062604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:32.374399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283914+0000","last_change":"2024-09-10T22:35:03.063406+0000","last_active":"2024-09-10T22:35:04.283914+0000","last_peered":"2024-09-10T22:35:04.283914+0000","last_clean":"2024-09-10T22:35:04.283914+0000","last_became_active":"2024-09-10T22:35:03.062960+0000","last_became_peered":"2024-09-10T22:35:03.062960+0000","last_unstale":"2024-09-10T22:35:04.283914+0000","last_undegraded":"2024-09-10T22:35:04.283914+0000","last_fullsized":"2024-09-10T22:35:04.283914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:08.845668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060881+0000","last_change":"2024-09-10T22:35:02.061366+0000","last_active":"2024-09-10T22:35:04.060881+0000","last_peered":"2024-09-10T22:35:04.060881+0000","last_clean":"2024-09-10T22:35:04.060881+0000","last_became_active":"2024-09-10T22:35:02.061092+0000","last_became_peered":"2024-09-10T22:35:02.061092+0000","last_unstale":"2024-09-10T22:35:04.060881+0000","last_undegraded":"2024-09-10T22:35:04.060881+0000","last_fullsized":"2024-09-10T22:35:04.060881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063828+0000","last_change":"2024-09-10T22:35:03.066189+0000","last_active":"2024-09-10T22:35:04.063828+0000","last_peered":"2024-09-10T22:35:04.063828+0000","last_clean":"2024-09-10T22:35:04.063828+0000","last_became_active":"2024-09-10T22:35:03.064797+0000","last_became_peered":"2024-09-10T22:35:03.064797+0000","last_unstale":"2024-09-10T22:35:04.063828+0000","last_undegraded":"2024-09-10T22:35:04.063828+0000","last_fullsized":"2024-09-10T22:35:04.063828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:33.379430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283111+0000","last_change":"2024-09-10T22:35:02.056652+0000","last_active":"2024-09-10T22:35:04.283111+0000","last_peered":"2024-09-10T22:35:04.283111+0000","last_clean":"2024-09-10T22:35:04.283111+0000","last_became_active":"2024-09-10T22:35:02.056381+0000","last_became_peered":"2024-09-10T22:35:02.056381+0000","last_unstale":"2024-09-10T22:35:04.283111+0000","last_undegraded":"2024-09-10T22:35:04.283111+0000","last_fullsized":"2024-09-10T22:35:04.283111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:15:42.884854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283576+0000","last_change":"2024-09-10T22:35:03.071792+0000","last_active":"2024-09-10T22:35:04.283576+0000","last_peered":"2024-09-10T22:35:04.283576+0000","last_clean":"2024-09-10T22:35:04.283576+0000","last_became_active":"2024-09-10T22:35:03.071456+0000","last_became_peered":"2024-09-10T22:35:03.071456+0000","last_unstale":"2024-09-10T22:35:04.283576+0000","last_undegraded":"2024-09-10T22:35:04.283576+0000","last_fullsized":"2024-09-10T22:35:04.283576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:35.334336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283948+0000","last_change":"2024-09-10T22:35:02.062689+0000","last_active":"2024-09-10T22:35:04.283948+0000","last_peered":"2024-09-10T22:35:04.283948+0000","last_clean":"2024-09-10T22:35:04.283948+0000","last_became_active":"2024-09-10T22:35:02.062395+0000","last_became_peered":"2024-09-10T22:35:02.062395+0000","last_unstale":"2024-09-10T22:35:04.283948+0000","last_undegraded":"2024-09-10T22:35:04.283948+0000","last_fullsized":"2024-09-10T22:35:04.283948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:10.653575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283121+0000","last_change":"2024-09-10T22:35:03.062742+0000","last_active":"2024-09-10T22:35:04.283121+0000","last_peered":"2024-09-10T22:35:04.283121+0000","last_clean":"2024-09-10T22:35:04.283121+0000","last_became_active":"2024-09-10T22:35:03.062374+0000","last_became_peered":"2024-09-10T22:35:03.062374+0000","last_unstale":"2024-09-10T22:35:04.283121+0000","last_undegraded":"2024-09-10T22:35:04.283121+0000","last_fullsized":"2024-09-10T22:35:04.283121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:26.571798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062695+0000","last_change":"2024-09-10T22:35:02.059477+0000","last_active":"2024-09-10T22:35:04.062695+0000","last_peered":"2024-09-10T22:35:04.062695+0000","last_clean":"2024-09-10T22:35:04.062695+0000","last_became_active":"2024-09-10T22:35:02.058863+0000","last_became_peered":"2024-09-10T22:35:02.058863+0000","last_unstale":"2024-09-10T22:35:04.062695+0000","last_undegraded":"2024-09-10T22:35:04.062695+0000","last_fullsized":"2024-09-10T22:35:04.062695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:44.446130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284043+0000","last_change":"2024-09-10T22:35:03.073670+0000","last_active":"2024-09-10T22:35:04.284043+0000","last_peered":"2024-09-10T22:35:04.284043+0000","last_clean":"2024-09-10T22:35:04.284043+0000","last_became_active":"2024-09-10T22:35:03.073362+0000","last_became_peered":"2024-09-10T22:35:03.073362+0000","last_unstale":"2024-09-10T22:35:04.284043+0000","last_undegraded":"2024-09-10T22:35:04.284043+0000","last_fullsized":"2024-09-10T22:35:04.284043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:56.758850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283238+0000","last_change":"2024-09-10T22:35:02.060433+0000","last_active":"2024-09-10T22:35:04.283238+0000","last_peered":"2024-09-10T22:35:04.283238+0000","last_clean":"2024-09-10T22:35:04.283238+0000","last_became_active":"2024-09-10T22:35:02.059798+0000","last_became_peered":"2024-09-10T22:35:02.059798+0000","last_unstale":"2024-09-10T22:35:04.283238+0000","last_undegraded":"2024-09-10T22:35:04.283238+0000","last_fullsized":"2024-09-10T22:35:04.283238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:07.281434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283176+0000","last_change":"2024-09-10T22:35:03.065328+0000","last_active":"2024-09-10T22:35:04.283176+0000","last_peered":"2024-09-10T22:35:04.283176+0000","last_clean":"2024-09-10T22:35:04.283176+0000","last_became_active":"2024-09-10T22:35:03.065050+0000","last_became_peered":"2024-09-10T22:35:03.065050+0000","last_unstale":"2024-09-10T22:35:04.283176+0000","last_undegraded":"2024-09-10T22:35:04.283176+0000","last_fullsized":"2024-09-10T22:35:04.283176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:28.267375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282601+0000","last_change":"2024-09-10T22:35:02.058914+0000","last_active":"2024-09-10T22:35:04.282601+0000","last_peered":"2024-09-10T22:35:04.282601+0000","last_clean":"2024-09-10T22:35:04.282601+0000","last_became_active":"2024-09-10T22:35:02.058054+0000","last_became_peered":"2024-09-10T22:35:02.058054+0000","last_unstale":"2024-09-10T22:35:04.282601+0000","last_undegraded":"2024-09-10T22:35:04.282601+0000","last_fullsized":"2024-09-10T22:35:04.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:07.958631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283220+0000","last_change":"2024-09-10T22:35:03.074096+0000","last_active":"2024-09-10T22:35:04.283220+0000","last_peered":"2024-09-10T22:35:04.283220+0000","last_clean":"2024-09-10T22:35:04.283220+0000","last_became_active":"2024-09-10T22:35:03.073454+0000","last_became_peered":"2024-09-10T22:35:03.073454+0000","last_unstale":"2024-09-10T22:35:04.283220+0000","last_undegraded":"2024-09-10T22:35:04.283220+0000","last_fullsized":"2024-09-10T22:35:04.283220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:16.644778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063831+0000","last_change":"2024-09-10T22:35:02.059067+0000","last_active":"2024-09-10T22:35:04.063831+0000","last_peered":"2024-09-10T22:35:04.063831+0000","last_clean":"2024-09-10T22:35:04.063831+0000","last_became_active":"2024-09-10T22:35:02.058800+0000","last_became_peered":"2024-09-10T22:35:02.058800+0000","last_unstale":"2024-09-10T22:35:04.063831+0000","last_undegraded":"2024-09-10T22:35:04.063831+0000","last_fullsized":"2024-09-10T22:35:04.063831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:32.586124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283832+0000","last_change":"2024-09-10T22:35:03.066866+0000","last_active":"2024-09-10T22:35:04.283832+0000","last_peered":"2024-09-10T22:35:04.283832+0000","last_clean":"2024-09-10T22:35:04.283832+0000","last_became_active":"2024-09-10T22:35:03.066570+0000","last_became_peered":"2024-09-10T22:35:03.066570+0000","last_unstale":"2024-09-10T22:35:04.283832+0000","last_undegraded":"2024-09-10T22:35:04.283832+0000","last_fullsized":"2024-09-10T22:35:04.283832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:57.797898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282840+0000","last_change":"2024-09-10T22:35:02.056593+0000","last_active":"2024-09-10T22:35:04.282840+0000","last_peered":"2024-09-10T22:35:04.282840+0000","last_clean":"2024-09-10T22:35:04.282840+0000","last_became_active":"2024-09-10T22:35:02.056253+0000","last_became_peered":"2024-09-10T22:35:02.056253+0000","last_unstale":"2024-09-10T22:35:04.282840+0000","last_undegraded":"2024-09-10T22:35:04.282840+0000","last_fullsized":"2024-09-10T22:35:04.282840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284206+0000","last_change":"2024-09-10T22:35:03.073877+0000","last_active":"2024-09-10T22:35:04.284206+0000","last_peered":"2024-09-10T22:35:04.284206+0000","last_clean":"2024-09-10T22:35:04.284206+0000","last_became_active":"2024-09-10T22:35:03.073479+0000","last_became_peered":"2024-09-10T22:35:03.073479+0000","last_unstale":"2024-09-10T22:35:04.284206+0000","last_undegraded":"2024-09-10T22:35:04.284206+0000","last_fullsized":"2024-09-10T22:35:04.284206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:31.691393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060976+0000","last_change":"2024-09-10T22:35:02.059186+0000","last_active":"2024-09-10T22:35:04.060976+0000","last_peered":"2024-09-10T22:35:04.060976+0000","last_clean":"2024-09-10T22:35:04.060976+0000","last_became_active":"2024-09-10T22:35:02.058905+0000","last_became_peered":"2024-09-10T22:35:02.058905+0000","last_unstale":"2024-09-10T22:35:04.060976+0000","last_undegraded":"2024-09-10T22:35:04.060976+0000","last_fullsized":"2024-09-10T22:35:04.060976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062968+0000","last_change":"2024-09-10T22:35:03.076118+0000","last_active":"2024-09-10T22:35:04.062968+0000","last_peered":"2024-09-10T22:35:04.062968+0000","last_clean":"2024-09-10T22:35:04.062968+0000","last_became_active":"2024-09-10T22:35:03.075621+0000","last_became_peered":"2024-09-10T22:35:03.075621+0000","last_unstale":"2024-09-10T22:35:04.062968+0000","last_undegraded":"2024-09-10T22:35:04.062968+0000","last_fullsized":"2024-09-10T22:35:04.062968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:48.372055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283210+0000","last_change":"2024-09-10T22:35:02.059788+0000","last_active":"2024-09-10T22:35:04.283210+0000","last_peered":"2024-09-10T22:35:04.283210+0000","last_clean":"2024-09-10T22:35:04.283210+0000","last_became_active":"2024-09-10T22:35:02.059419+0000","last_became_peered":"2024-09-10T22:35:02.059419+0000","last_unstale":"2024-09-10T22:35:04.283210+0000","last_undegraded":"2024-09-10T22:35:04.283210+0000","last_fullsized":"2024-09-10T22:35:04.283210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:05.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283722+0000","last_change":"2024-09-10T22:35:03.069011+0000","last_active":"2024-09-10T22:35:04.283722+0000","last_peered":"2024-09-10T22:35:04.283722+0000","last_clean":"2024-09-10T22:35:04.283722+0000","last_became_active":"2024-09-10T22:35:03.068736+0000","last_became_peered":"2024-09-10T22:35:03.068736+0000","last_unstale":"2024-09-10T22:35:04.283722+0000","last_undegraded":"2024-09-10T22:35:04.283722+0000","last_fullsized":"2024-09-10T22:35:04.283722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:28.595416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282999+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:04.282999+0000","last_peered":"2024-09-10T22:35:04.282999+0000","last_clean":"2024-09-10T22:35:04.282999+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:04.282999+0000","last_undegraded":"2024-09-10T22:35:04.282999+0000","last_fullsized":"2024-09-10T22:35:04.282999+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.061890+0000","last_change":"2024-09-10T22:35:02.057812+0000","last_active":"2024-09-10T22:35:04.061890+0000","last_peered":"2024-09-10T22:35:04.061890+0000","last_clean":"2024-09-10T22:35:04.061890+0000","last_became_active":"2024-09-10T22:35:02.057415+0000","last_became_peered":"2024-09-10T22:35:02.057415+0000","last_unstale":"2024-09-10T22:35:04.061890+0000","last_undegraded":"2024-09-10T22:35:04.061890+0000","last_fullsized":"2024-09-10T22:35:04.061890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:28.670744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283212+0000","last_change":"2024-09-10T22:35:03.079281+0000","last_active":"2024-09-10T22:35:04.283212+0000","last_peered":"2024-09-10T22:35:04.283212+0000","last_clean":"2024-09-10T22:35:04.283212+0000","last_became_active":"2024-09-10T22:35:03.078916+0000","last_became_peered":"2024-09-10T22:35:03.078916+0000","last_unstale":"2024-09-10T22:35:04.283212+0000","last_undegraded":"2024-09-10T22:35:04.283212+0000","last_fullsized":"2024-09-10T22:35:04.283212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:25.669306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282681+0000","last_change":"2024-09-10T22:35:02.065966+0000","last_active":"2024-09-10T22:35:04.282681+0000","last_peered":"2024-09-10T22:35:04.282681+0000","last_clean":"2024-09-10T22:35:04.282681+0000","last_became_active":"2024-09-10T22:35:02.065658+0000","last_became_peered":"2024-09-10T22:35:02.065658+0000","last_unstale":"2024-09-10T22:35:04.282681+0000","last_undegraded":"2024-09-10T22:35:04.282681+0000","last_fullsized":"2024-09-10T22:35:04.282681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:39.077105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282773+0000","last_change":"2024-09-10T22:35:03.055020+0000","last_active":"2024-09-10T22:35:04.282773+0000","last_peered":"2024-09-10T22:35:04.282773+0000","last_clean":"2024-09-10T22:35:04.282773+0000","last_became_active":"2024-09-10T22:35:03.052854+0000","last_became_peered":"2024-09-10T22:35:03.052854+0000","last_unstale":"2024-09-10T22:35:04.282773+0000","last_undegraded":"2024-09-10T22:35:04.282773+0000","last_fullsized":"2024-09-10T22:35:04.282773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:40.082137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283032+0000","last_change":"2024-09-10T22:35:02.057593+0000","last_active":"2024-09-10T22:35:04.283032+0000","last_peered":"2024-09-10T22:35:04.283032+0000","last_clean":"2024-09-10T22:35:04.283032+0000","last_became_active":"2024-09-10T22:35:02.057203+0000","last_became_peered":"2024-09-10T22:35:02.057203+0000","last_unstale":"2024-09-10T22:35:04.283032+0000","last_undegraded":"2024-09-10T22:35:04.283032+0000","last_fullsized":"2024-09-10T22:35:04.283032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:49.760894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060660+0000","last_change":"2024-09-10T22:35:03.057957+0000","last_active":"2024-09-10T22:35:04.060660+0000","last_peered":"2024-09-10T22:35:04.060660+0000","last_clean":"2024-09-10T22:35:04.060660+0000","last_became_active":"2024-09-10T22:35:03.057658+0000","last_became_peered":"2024-09-10T22:35:03.057658+0000","last_unstale":"2024-09-10T22:35:04.060660+0000","last_undegraded":"2024-09-10T22:35:04.060660+0000","last_fullsized":"2024-09-10T22:35:04.060660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:10.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062941+0000","last_change":"2024-09-10T22:35:02.059793+0000","last_active":"2024-09-10T22:35:04.062941+0000","last_peered":"2024-09-10T22:35:04.062941+0000","last_clean":"2024-09-10T22:35:04.062941+0000","last_became_active":"2024-09-10T22:35:02.059208+0000","last_became_peered":"2024-09-10T22:35:02.059208+0000","last_unstale":"2024-09-10T22:35:04.062941+0000","last_undegraded":"2024-09-10T22:35:04.062941+0000","last_fullsized":"2024-09-10T22:35:04.062941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:21.383738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.063522+0000","last_change":"2024-09-10T22:35:03.067755+0000","last_active":"2024-09-10T22:35:04.063522+0000","last_peered":"2024-09-10T22:35:04.063522+0000","last_clean":"2024-09-10T22:35:04.063522+0000","last_became_active":"2024-09-10T22:35:03.067489+0000","last_became_peered":"2024-09-10T22:35:03.067489+0000","last_unstale":"2024-09-10T22:35:04.063522+0000","last_undegraded":"2024-09-10T22:35:04.063522+0000","last_fullsized":"2024-09-10T22:35:04.063522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:22.388769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060864+0000","last_change":"2024-09-10T22:35:02.063524+0000","last_active":"2024-09-10T22:35:04.060864+0000","last_peered":"2024-09-10T22:35:04.060864+0000","last_clean":"2024-09-10T22:35:04.060864+0000","last_became_active":"2024-09-10T22:35:02.063232+0000","last_became_peered":"2024-09-10T22:35:02.063232+0000","last_unstale":"2024-09-10T22:35:04.060864+0000","last_undegraded":"2024-09-10T22:35:04.060864+0000","last_fullsized":"2024-09-10T22:35:04.060864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:41.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062991+0000","last_change":"2024-09-10T22:35:03.071953+0000","last_active":"2024-09-10T22:35:04.062991+0000","last_peered":"2024-09-10T22:35:04.062991+0000","last_clean":"2024-09-10T22:35:04.062991+0000","last_became_active":"2024-09-10T22:35:03.071660+0000","last_became_peered":"2024-09-10T22:35:03.071660+0000","last_unstale":"2024-09-10T22:35:04.062991+0000","last_undegraded":"2024-09-10T22:35:04.062991+0000","last_fullsized":"2024-09-10T22:35:04.062991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:15.659782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284091+0000","last_change":"2024-09-10T22:35:02.067388+0000","last_active":"2024-09-10T22:35:04.284091+0000","last_peered":"2024-09-10T22:35:04.284091+0000","last_clean":"2024-09-10T22:35:04.284091+0000","last_became_active":"2024-09-10T22:35:02.067088+0000","last_became_peered":"2024-09-10T22:35:02.067088+0000","last_unstale":"2024-09-10T22:35:04.284091+0000","last_undegraded":"2024-09-10T22:35:04.284091+0000","last_fullsized":"2024-09-10T22:35:04.284091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:36.542782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060999+0000","last_change":"2024-09-10T22:35:03.058800+0000","last_active":"2024-09-10T22:35:04.060999+0000","last_peered":"2024-09-10T22:35:04.060999+0000","last_clean":"2024-09-10T22:35:04.060999+0000","last_became_active":"2024-09-10T22:35:03.058503+0000","last_became_peered":"2024-09-10T22:35:03.058503+0000","last_unstale":"2024-09-10T22:35:04.060999+0000","last_undegraded":"2024-09-10T22:35:04.060999+0000","last_fullsized":"2024-09-10T22:35:04.060999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.058112+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.057838+0000","last_became_peered":"2024-09-10T22:35:02.057838+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:20.966298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283672+0000","last_change":"2024-09-10T22:35:03.072054+0000","last_active":"2024-09-10T22:35:04.283672+0000","last_peered":"2024-09-10T22:35:04.283672+0000","last_clean":"2024-09-10T22:35:04.283672+0000","last_became_active":"2024-09-10T22:35:03.071739+0000","last_became_peered":"2024-09-10T22:35:03.071739+0000","last_unstale":"2024-09-10T22:35:04.283672+0000","last_undegraded":"2024-09-10T22:35:04.283672+0000","last_fullsized":"2024-09-10T22:35:04.283672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:57.176991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283676+0000","last_change":"2024-09-10T22:35:02.064151+0000","last_active":"2024-09-10T22:35:04.283676+0000","last_peered":"2024-09-10T22:35:04.283676+0000","last_clean":"2024-09-10T22:35:04.283676+0000","last_became_active":"2024-09-10T22:35:02.063573+0000","last_became_peered":"2024-09-10T22:35:02.063573+0000","last_unstale":"2024-09-10T22:35:04.283676+0000","last_undegraded":"2024-09-10T22:35:04.283676+0000","last_fullsized":"2024-09-10T22:35:04.283676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:21.244402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283654+0000","last_change":"2024-09-10T22:35:03.057069+0000","last_active":"2024-09-10T22:35:04.283654+0000","last_peered":"2024-09-10T22:35:04.283654+0000","last_clean":"2024-09-10T22:35:04.283654+0000","last_became_active":"2024-09-10T22:35:03.056760+0000","last_became_peered":"2024-09-10T22:35:03.056760+0000","last_unstale":"2024-09-10T22:35:04.283654+0000","last_undegraded":"2024-09-10T22:35:04.283654+0000","last_fullsized":"2024-09-10T22:35:04.283654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:11.516935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.282652+0000","last_change":"2024-09-10T22:35:02.066275+0000","last_active":"2024-09-10T22:35:04.282652+0000","last_peered":"2024-09-10T22:35:04.282652+0000","last_clean":"2024-09-10T22:35:04.282652+0000","last_became_active":"2024-09-10T22:35:02.065887+0000","last_became_peered":"2024-09-10T22:35:02.065887+0000","last_unstale":"2024-09-10T22:35:04.282652+0000","last_undegraded":"2024-09-10T22:35:04.282652+0000","last_fullsized":"2024-09-10T22:35:04.282652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:09.305553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062290+0000","last_change":"2024-09-10T22:35:03.054840+0000","last_active":"2024-09-10T22:35:04.062290+0000","last_peered":"2024-09-10T22:35:04.062290+0000","last_clean":"2024-09-10T22:35:04.062290+0000","last_became_active":"2024-09-10T22:35:03.054562+0000","last_became_peered":"2024-09-10T22:35:03.054562+0000","last_unstale":"2024-09-10T22:35:04.062290+0000","last_undegraded":"2024-09-10T22:35:04.062290+0000","last_fullsized":"2024-09-10T22:35:04.062290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:29.675776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060764+0000","last_change":"2024-09-10T22:35:02.061273+0000","last_active":"2024-09-10T22:35:04.060764+0000","last_peered":"2024-09-10T22:35:04.060764+0000","last_clean":"2024-09-10T22:35:04.060764+0000","last_became_active":"2024-09-10T22:35:02.060953+0000","last_became_peered":"2024-09-10T22:35:02.060953+0000","last_unstale":"2024-09-10T22:35:04.060764+0000","last_undegraded":"2024-09-10T22:35:04.060764+0000","last_fullsized":"2024-09-10T22:35:04.060764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:32.172943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284309+0000","last_change":"2024-09-10T22:35:03.078424+0000","last_active":"2024-09-10T22:35:04.284309+0000","last_peered":"2024-09-10T22:35:04.284309+0000","last_clean":"2024-09-10T22:35:04.284309+0000","last_became_active":"2024-09-10T22:35:03.078153+0000","last_became_peered":"2024-09-10T22:35:03.078153+0000","last_unstale":"2024-09-10T22:35:04.284309+0000","last_undegraded":"2024-09-10T22:35:04.284309+0000","last_fullsized":"2024-09-10T22:35:04.284309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:48.985804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.060797+0000","last_change":"2024-09-10T22:35:02.061749+0000","last_active":"2024-09-10T22:35:04.060797+0000","last_peered":"2024-09-10T22:35:04.060797+0000","last_clean":"2024-09-10T22:35:04.060797+0000","last_became_active":"2024-09-10T22:35:02.061248+0000","last_became_peered":"2024-09-10T22:35:02.061248+0000","last_unstale":"2024-09-10T22:35:04.060797+0000","last_undegraded":"2024-09-10T22:35:04.060797+0000","last_fullsized":"2024-09-10T22:35:04.060797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:59:49.960770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283758+0000","last_change":"2024-09-10T22:35:03.076736+0000","last_active":"2024-09-10T22:35:04.283758+0000","last_peered":"2024-09-10T22:35:04.283758+0000","last_clean":"2024-09-10T22:35:04.283758+0000","last_became_active":"2024-09-10T22:35:03.075379+0000","last_became_peered":"2024-09-10T22:35:03.075379+0000","last_unstale":"2024-09-10T22:35:04.283758+0000","last_undegraded":"2024-09-10T22:35:04.283758+0000","last_fullsized":"2024-09-10T22:35:04.283758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:58.399411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.284154+0000","last_change":"2024-09-10T22:35:02.065909+0000","last_active":"2024-09-10T22:35:04.284154+0000","last_peered":"2024-09-10T22:35:04.284154+0000","last_clean":"2024-09-10T22:35:04.284154+0000","last_became_active":"2024-09-10T22:35:02.065539+0000","last_became_peered":"2024-09-10T22:35:02.065539+0000","last_unstale":"2024-09-10T22:35:04.284154+0000","last_undegraded":"2024-09-10T22:35:04.284154+0000","last_fullsized":"2024-09-10T22:35:04.284154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:32.374399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283186+0000","last_change":"2024-09-10T22:35:03.060477+0000","last_active":"2024-09-10T22:35:04.283186+0000","last_peered":"2024-09-10T22:35:04.283186+0000","last_clean":"2024-09-10T22:35:04.283186+0000","last_became_active":"2024-09-10T22:35:03.060190+0000","last_became_peered":"2024-09-10T22:35:03.060190+0000","last_unstale":"2024-09-10T22:35:04.283186+0000","last_undegraded":"2024-09-10T22:35:04.283186+0000","last_fullsized":"2024-09-10T22:35:04.283186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:56.641563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062898+0000","last_change":"2024-09-10T22:35:03.060225+0000","last_active":"2024-09-10T22:35:04.062898+0000","last_peered":"2024-09-10T22:35:04.062898+0000","last_clean":"2024-09-10T22:35:04.062898+0000","last_became_active":"2024-09-10T22:35:03.059807+0000","last_became_peered":"2024-09-10T22:35:03.059807+0000","last_unstale":"2024-09-10T22:35:04.062898+0000","last_undegraded":"2024-09-10T22:35:04.062898+0000","last_fullsized":"2024-09-10T22:35:04.062898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:12.132002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283516+0000","last_change":"2024-09-10T22:35:02.066541+0000","last_active":"2024-09-10T22:35:04.283516+0000","last_peered":"2024-09-10T22:35:04.283516+0000","last_clean":"2024-09-10T22:35:04.283516+0000","last_became_active":"2024-09-10T22:35:02.066259+0000","last_became_peered":"2024-09-10T22:35:02.066259+0000","last_unstale":"2024-09-10T22:35:04.283516+0000","last_undegraded":"2024-09-10T22:35:04.283516+0000","last_fullsized":"2024-09-10T22:35:04.283516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:58.279821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.062332+0000","last_change":"2024-09-10T22:35:03.068676+0000","last_active":"2024-09-10T22:35:04.062332+0000","last_peered":"2024-09-10T22:35:04.062332+0000","last_clean":"2024-09-10T22:35:04.062332+0000","last_became_active":"2024-09-10T22:35:03.068384+0000","last_became_peered":"2024-09-10T22:35:03.068384+0000","last_unstale":"2024-09-10T22:35:04.062332+0000","last_undegraded":"2024-09-10T22:35:04.062332+0000","last_fullsized":"2024-09-10T22:35:04.062332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:56.275057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283838+0000","last_change":"2024-09-10T22:35:02.065459+0000","last_active":"2024-09-10T22:35:04.283838+0000","last_peered":"2024-09-10T22:35:04.283838+0000","last_clean":"2024-09-10T22:35:04.283838+0000","last_became_active":"2024-09-10T22:35:02.065145+0000","last_became_peered":"2024-09-10T22:35:02.065145+0000","last_unstale":"2024-09-10T22:35:04.283838+0000","last_undegraded":"2024-09-10T22:35:04.283838+0000","last_fullsized":"2024-09-10T22:35:04.283838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:21.383738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283725+0000","last_change":"2024-09-10T22:35:03.063955+0000","last_active":"2024-09-10T22:35:04.283725+0000","last_peered":"2024-09-10T22:35:04.283725+0000","last_clean":"2024-09-10T22:35:04.283725+0000","last_became_active":"2024-09-10T22:35:03.063679+0000","last_became_peered":"2024-09-10T22:35:03.063679+0000","last_unstale":"2024-09-10T22:35:04.283725+0000","last_undegraded":"2024-09-10T22:35:04.283725+0000","last_fullsized":"2024-09-10T22:35:04.283725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:56.641563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283509+0000","last_change":"2024-09-10T22:35:02.062827+0000","last_active":"2024-09-10T22:35:04.283509+0000","last_peered":"2024-09-10T22:35:04.283509+0000","last_clean":"2024-09-10T22:35:04.283509+0000","last_became_active":"2024-09-10T22:35:02.062342+0000","last_became_peered":"2024-09-10T22:35:02.062342+0000","last_unstale":"2024-09-10T22:35:04.283509+0000","last_undegraded":"2024-09-10T22:35:04.283509+0000","last_fullsized":"2024-09-10T22:35:04.283509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:36.987666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283072+0000","last_change":"2024-09-10T22:35:03.080815+0000","last_active":"2024-09-10T22:35:04.283072+0000","last_peered":"2024-09-10T22:35:04.283072+0000","last_clean":"2024-09-10T22:35:04.283072+0000","last_became_active":"2024-09-10T22:35:03.080484+0000","last_became_peered":"2024-09-10T22:35:03.080484+0000","last_unstale":"2024-09-10T22:35:04.283072+0000","last_undegraded":"2024-09-10T22:35:04.283072+0000","last_fullsized":"2024-09-10T22:35:04.283072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:02.037934+0000","last_clean_scrub_stamp":"2024-09-10T22:35:02.037934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:28:58.700652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T22:35:04.283792+0000","last_change":"2024-09-10T22:35:02.065788+0000","last_active":"2024-09-10T22:35:04.283792+0000","last_peered":"2024-09-10T22:35:04.283792+0000","last_clean":"2024-09-10T22:35:04.283792+0000","last_became_active":"2024-09-10T22:35:02.065500+0000","last_became_peered":"2024-09-10T22:35:02.065500+0000","last_unstale":"2024-09-10T22:35:04.283792+0000","last_undegraded":"2024-09-10T22:35:04.283792+0000","last_fullsized":"2024-09-10T22:35:04.283792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:01.032902+0000","last_clean_scrub_stamp":"2024-09-10T22:35:01.032902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:10.573361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":663271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":212992,"data_stored":72411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":72447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":815104,"data_stored":663327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":72321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:16.610 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:35:16.611 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T22:35:16.611 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T22:35:16.611 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T22:35:16.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.750+0000 7ffbd192b640 1 -- 172.21.15.66:0/869302256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbb40051c0 msgr2=0x7ffbb400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.750+0000 7ffbd192b640 1 --2- 172.21.15.66:0/869302256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbb40051c0 0x7ffbb400b9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40016c0 tx=0x7ffbb4005830 comp rx=0 tx=0).stop 2024-09-10T22:35:16.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.750+0000 7ffbd192b640 1 -- 172.21.15.66:0/869302256 shutdown_connections 2024-09-10T22:35:16.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.750+0000 7ffbd192b640 1 --2- 172.21.15.66:0/869302256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbb40051c0 0x7ffbb400b9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.750+0000 7ffbd192b640 1 -- 172.21.15.66:0/869302256 >> 172.21.15.66:0/869302256 conn(0x7ffbcc09d260 msgr2=0x7ffbcc0d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:16.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.750+0000 7ffbd192b640 1 -- 172.21.15.66:0/869302256 shutdown_connections 2024-09-10T22:35:16.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.750+0000 7ffbd192b640 1 -- 172.21.15.66:0/869302256 wait complete. 2024-09-10T22:35:16.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.752+0000 7ffbd192b640 1 Processor -- start 2024-09-10T22:35:16.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.752+0000 7ffbd192b640 1 -- start start 2024-09-10T22:35:16.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb40051c0 0x7ffbcc1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbcc1f1f70 0x7ffbcc0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc0b2dd0 0x7ffbcc1f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc161150 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc0b4130 con 0x7ffbb40051c0 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc1612d0 con 0x7ffbcc1f1f70 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc0b2dd0 0x7ffbcc1f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc0b2dd0 0x7ffbcc1f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35942/0 (socket says 172.21.15.66:35942) 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 -- 172.21.15.66:0/3690533582 learned_addr learned my addr 172.21.15.66:0/3690533582 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcaffd640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb40051c0 0x7ffbcc1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbca7fc640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbcc1f1f70 0x7ffbcc0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 -- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbcc1f1f70 msgr2=0x7ffbcc0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbcc1f1f70 0x7ffbcc0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 -- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb40051c0 msgr2=0x7ffbcc1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb40051c0 0x7ffbcc1f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 -- 172.21.15.66:0/3690533582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc0b10a0 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbcb7fe640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc0b2dd0 0x7ffbcc1f81c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402a930 tx=0x7ffbc4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbb3fff640 1 -- 172.21.15.66:0/3690533582 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4037070 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbb3fff640 1 -- 172.21.15.66:0/3690533582 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbc4003ba0 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc0b1380 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbb3fff640 1 -- 172.21.15.66:0/3690533582 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40012e0 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.753+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc0b1940 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.755+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbcc1617c0 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.755+0000 7ffbb3fff640 1 -- 172.21.15.66:0/3690533582 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffbc403d470 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.756+0000 7ffbb3fff640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffbb8077fa0 0x7ffbb807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:16.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.756+0000 7ffbb3fff640 1 -- 172.21.15.66:0/3690533582 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffbc40b9180 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.756+0000 7ffbcaffd640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffbb8077fa0 0x7ffbb807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:16.759 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.756+0000 7ffbcaffd640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffbb8077fa0 0x7ffbb807a460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005fd0 tx=0x7ffbbc009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:16.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.760+0000 7ffbb3fff640 1 -- 172.21.15.66:0/3690533582 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ffbc4044020 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.982+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ffbcc08d6e0 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.982+0000 7ffbb3fff640 1 -- 172.21.15.66:0/3690533582 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ffbc4081d40 con 0x7ffbcc0b2dd0 2024-09-10T22:35:16.983 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:16.983 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T22:35:16.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffbb8077fa0 msgr2=0x7ffbb807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffbb8077fa0 0x7ffbb807a460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005fd0 tx=0x7ffbbc009500 comp rx=0 tx=0).stop 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc0b2dd0 msgr2=0x7ffbcc1f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc0b2dd0 0x7ffbcc1f81c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffbc402a930 tx=0x7ffbc4003820 comp rx=0 tx=0).stop 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 shutdown_connections 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffbb8077fa0 0x7ffbb807a460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffbcc0b2dd0 0x7ffbcc1f81c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffbcc1f1f70 0x7ffbcc0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 --2- 172.21.15.66:0/3690533582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb40051c0 0x7ffbcc1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 >> 172.21.15.66:0/3690533582 conn(0x7ffbcc09d260 msgr2=0x7ffbcc06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 shutdown_connections 2024-09-10T22:35:16.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:16.984+0000 7ffbd192b640 1 -- 172.21.15.66:0/3690533582 wait complete. 2024-09-10T22:35:16.996 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T22:35:16.996 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:17.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 -- 172.21.15.66:0/2670939994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50155270 msgr2=0x7f1f500ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2670939994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50155270 0x7f1f500ce860 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f500746e0 tx=0x7f1f4002a5f0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 -- 172.21.15.66:0/2670939994 shutdown_connections 2024-09-10T22:35:17.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2670939994 >> v2:172.21.15.94:3300/0 conn(0x7f1f500ceda0 0x7f1f50072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2670939994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50155270 0x7f1f500ce860 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 -- 172.21.15.66:0/2670939994 >> 172.21.15.66:0/2670939994 conn(0x7f1f500b16d0 msgr2=0x7f1f500b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:17.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 -- 172.21.15.66:0/2670939994 shutdown_connections 2024-09-10T22:35:17.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.101+0000 7f1f57efe640 1 -- 172.21.15.66:0/2670939994 wait complete. 2024-09-10T22:35:17.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.102+0000 7f1f57efe640 1 Processor -- start 2024-09-10T22:35:17.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.102+0000 7f1f57efe640 1 -- start start 2024-09-10T22:35:17.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f57efe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f500ceda0 0x7f1f500c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f57efe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50154e90 0x7f1f50152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f57efe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f50155270 0x7f1f50159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f57efe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500733a0 con 0x7f1f50154e90 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f57efe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1f50073220 con 0x7f1f50155270 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f57efe640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f1f50073520 con 0x7f1f500ceda0 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f55499640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50154e90 0x7f1f50152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f55499640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50154e90 0x7f1f50152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35958/0 (socket says 172.21.15.66:35958) 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f55499640 1 -- 172.21.15.66:0/2520967336 learned_addr learned my addr 172.21.15.66:0/2520967336 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f5649b640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f50155270 0x7f1f50159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f55c9a640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f500ceda0 0x7f1f500c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f55499640 1 -- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f500ceda0 msgr2=0x7f1f500c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.103+0000 7f1f55499640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f500ceda0 0x7f1f500c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f55499640 1 -- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f50155270 msgr2=0x7f1f50159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f55499640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f50155270 0x7f1f50159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f55499640 1 -- 172.21.15.66:0/2520967336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f50152b40 con 0x7f1f50154e90 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f55499640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50154e90 0x7f1f50152480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034040 tx=0x7f1f40003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f467fc640 1 -- 172.21.15.66:0/2520967336 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4003e030 con 0x7f1f50154e90 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f467fc640 1 -- 172.21.15.66:0/2520967336 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f40003d10 con 0x7f1f50154e90 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f50152e20 con 0x7f1f50154e90 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f467fc640 1 -- 172.21.15.66:0/2520967336 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40035040 con 0x7f1f50154e90 2024-09-10T22:35:17.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.104+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f501533e0 con 0x7f1f50154e90 2024-09-10T22:35:17.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.105+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f500cb120 con 0x7f1f50154e90 2024-09-10T22:35:17.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.106+0000 7f1f467fc640 1 -- 172.21.15.66:0/2520967336 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1f400049c0 con 0x7f1f50154e90 2024-09-10T22:35:17.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.106+0000 7f1f467fc640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1f34077fa0 0x7f1f3407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.106+0000 7f1f467fc640 1 -- 172.21.15.66:0/2520967336 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1f400ba4e0 con 0x7f1f50154e90 2024-09-10T22:35:17.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.106+0000 7f1f55c9a640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1f34077fa0 0x7f1f3407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.107+0000 7f1f55c9a640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1f34077fa0 0x7f1f3407a460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1f5014b5f0 tx=0x7f1f4c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:17.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.110+0000 7f1f467fc640 1 -- 172.21.15.66:0/2520967336 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1f40082a20 con 0x7f1f50154e90 2024-09-10T22:35:17.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.332+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f500b5ae0 con 0x7f1f50154e90 2024-09-10T22:35:17.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.333+0000 7f1f467fc640 1 -- 172.21.15.66:0/2520967336 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f1f40087450 con 0x7f1f50154e90 2024-09-10T22:35:17.334 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:17.334 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:17.335 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1f34077fa0 msgr2=0x7f1f3407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1f34077fa0 0x7f1f3407a460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1f5014b5f0 tx=0x7f1f4c009040 comp rx=0 tx=0).stop 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50154e90 msgr2=0x7f1f50152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50154e90 0x7f1f50152480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034040 tx=0x7f1f40003820 comp rx=0 tx=0).stop 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 shutdown_connections 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1f34077fa0 0x7f1f3407a460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f50155270 0x7f1f50159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f50154e90 0x7f1f50152480 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 --2- 172.21.15.66:0/2520967336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f1f500ceda0 0x7f1f500c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 >> 172.21.15.66:0/2520967336 conn(0x7f1f500b16d0 msgr2=0x7f1f500cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 shutdown_connections 2024-09-10T22:35:17.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.335+0000 7f1f57efe640 1 -- 172.21.15.66:0/2520967336 wait complete. 2024-09-10T22:35:17.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:17.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 -- 172.21.15.66:0/2116903096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d2a10 msgr2=0x7fbad40ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/2116903096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d2a10 0x7fbad40ceab0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbad408c290 tx=0x7fbacc02a560 comp rx=0 tx=0).stop 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 -- 172.21.15.66:0/2116903096 shutdown_connections 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/2116903096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d2a10 0x7fbad40ceab0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/2116903096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad40d1ae0 0x7fbad40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/2116903096 >> v2:172.21.15.94:3300/0 conn(0x7fbad415e090 0x7fbad40d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 -- 172.21.15.66:0/2116903096 >> 172.21.15.66:0/2116903096 conn(0x7fbad40d3f10 msgr2=0x7fbad40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 -- 172.21.15.66:0/2116903096 shutdown_connections 2024-09-10T22:35:17.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.451+0000 7fbad9ecd640 1 -- 172.21.15.66:0/2116903096 wait complete. 2024-09-10T22:35:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.452+0000 7fbad9ecd640 1 Processor -- start 2024-09-10T22:35:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad9ecd640 1 -- start start 2024-09-10T22:35:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad9ecd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d1ae0 0x7fbad40cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad9ecd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40d2a10 0x7fbad4157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad9ecd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad415e090 0x7fbad4158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad9ecd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40cf380 con 0x7fbad40d1ae0 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad9ecd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40cf200 con 0x7fbad40d2a10 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad9ecd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbad40cf500 con 0x7fbad415e090 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d1ae0 0x7fbad40cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d1ae0 0x7fbad40cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35978/0 (socket says 172.21.15.66:35978) 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.453+0000 7fbad37fe640 1 -- 172.21.15.66:0/3477862871 learned_addr learned my addr 172.21.15.66:0/3477862871 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad3fff640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad415e090 0x7fbad4158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.456 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad2ffd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40d2a10 0x7fbad4157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad37fe640 1 -- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad415e090 msgr2=0x7fbad4158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad37fe640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad415e090 0x7fbad4158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad37fe640 1 -- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40d2a10 msgr2=0x7fbad4157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad37fe640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40d2a10 0x7fbad4157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad37fe640 1 -- 172.21.15.66:0/3477862871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad40cc650 con 0x7fbad40d1ae0 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad37fe640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d1ae0 0x7fbad40cc110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbac400dae0 tx=0x7fbac400dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbacbfff640 1 -- 172.21.15.66:0/3477862871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4017070 con 0x7fbad40d1ae0 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbacbfff640 1 -- 172.21.15.66:0/3477862871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbac4003970 con 0x7fbad40d1ae0 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbacbfff640 1 -- 172.21.15.66:0/3477862871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4003dc0 con 0x7fbad40d1ae0 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad41f8aa0 con 0x7fbad40d1ae0 2024-09-10T22:35:17.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.454+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41f8200 con 0x7fbad40d1ae0 2024-09-10T22:35:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.455+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad40d24d0 con 0x7fbad40d1ae0 2024-09-10T22:35:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.456+0000 7fbacbfff640 1 -- 172.21.15.66:0/3477862871 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbac4004020 con 0x7fbad40d1ae0 2024-09-10T22:35:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.456+0000 7fbacbfff640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbab4077f70 0x7fbab407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.456+0000 7fbacbfff640 1 -- 172.21.15.66:0/3477862871 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbac409a310 con 0x7fbad40d1ae0 2024-09-10T22:35:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.457+0000 7fbad2ffd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbab4077f70 0x7fbab407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.457+0000 7fbad2ffd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbab4077f70 0x7fbab407a430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbad406e980 tx=0x7fbabc00b310 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:17.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.461+0000 7fbacbfff640 1 -- 172.21.15.66:0/3477862871 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbac4020030 con 0x7fbad40d1ae0 2024-09-10T22:35:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.680+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbad40ce260 con 0x7fbad40d1ae0 2024-09-10T22:35:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.680+0000 7fbacbfff640 1 -- 172.21.15.66:0/3477862871 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7fbac4062e10 con 0x7fbad40d1ae0 2024-09-10T22:35:17.681 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:17.682 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbab4077f70 msgr2=0x7fbab407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbab4077f70 0x7fbab407a430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbad406e980 tx=0x7fbabc00b310 comp rx=0 tx=0).stop 2024-09-10T22:35:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d1ae0 msgr2=0x7fbad40cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d1ae0 0x7fbad40cc110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbac400dae0 tx=0x7fbac400dee0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 shutdown_connections 2024-09-10T22:35:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbab4077f70 0x7fbab407a430 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbad415e090 0x7fbad4158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbad40d2a10 0x7fbad4157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 --2- 172.21.15.66:0/3477862871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbad40d1ae0 0x7fbad40cc110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.682+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 >> 172.21.15.66:0/3477862871 conn(0x7fbad40d3f10 msgr2=0x7fbad40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.683+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 shutdown_connections 2024-09-10T22:35:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.683+0000 7fbad9ecd640 1 -- 172.21.15.66:0/3477862871 wait complete. 2024-09-10T22:35:17.695 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:17.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 -- 172.21.15.66:0/816668637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c0ceda0 msgr2=0x7fd06c072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 --2- 172.21.15.66:0/816668637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c0ceda0 0x7fd06c072ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd06c06e820 tx=0x7fd06402aac0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 -- 172.21.15.66:0/816668637 shutdown_connections 2024-09-10T22:35:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 --2- 172.21.15.66:0/816668637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c0ceda0 0x7fd06c072ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 --2- 172.21.15.66:0/816668637 >> v2:172.21.15.94:3300/0 conn(0x7fd06c155270 0x7fd06c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 -- 172.21.15.66:0/816668637 >> 172.21.15.66:0/816668637 conn(0x7fd06c0b16d0 msgr2=0x7fd06c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 -- 172.21.15.66:0/816668637 shutdown_connections 2024-09-10T22:35:17.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.798+0000 7fd073ed4640 1 -- 172.21.15.66:0/816668637 wait complete. 2024-09-10T22:35:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.799+0000 7fd073ed4640 1 Processor -- start 2024-09-10T22:35:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.799+0000 7fd073ed4640 1 -- start start 2024-09-10T22:35:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd073ed4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd06c0ceda0 0x7fd06c1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd073ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c154e90 0x7fd06c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd073ed4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c155270 0x7fd06c148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd073ed4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c0733a0 con 0x7fd06c155270 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd073ed4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c073220 con 0x7fd06c154e90 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd073ed4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c073520 con 0x7fd06c0ceda0 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd072471640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c155270 0x7fd06c148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd072471640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c155270 0x7fd06c148f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36008/0 (socket says 172.21.15.66:36008) 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd072471640 1 -- 172.21.15.66:0/2165713903 learned_addr learned my addr 172.21.15.66:0/2165713903 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd07146f640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c154e90 0x7fd06c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.800+0000 7fd071c70640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd06c0ceda0 0x7fd06c1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd072471640 1 -- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd06c0ceda0 msgr2=0x7fd06c1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd072471640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd06c0ceda0 0x7fd06c1f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd072471640 1 -- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c154e90 msgr2=0x7fd06c0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd072471640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c154e90 0x7fd06c0b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd072471640 1 -- 172.21.15.66:0/2165713903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c0b5460 con 0x7fd06c155270 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd072471640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c155270 0x7fd06c148f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd064009d60 tx=0x7fd064003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd05e7fc640 1 -- 172.21.15.66:0/2165713903 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064044070 con 0x7fd06c155270 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd05e7fc640 1 -- 172.21.15.66:0/2165713903 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd064004250 con 0x7fd06c155270 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c1f5530 con 0x7fd06c155270 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd05e7fc640 1 -- 172.21.15.66:0/2165713903 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064004680 con 0x7fd06c155270 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.801+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c1f4f70 con 0x7fd06c155270 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.802+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06c073a40 con 0x7fd06c155270 2024-09-10T22:35:17.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.803+0000 7fd05e7fc640 1 -- 172.21.15.66:0/2165713903 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd06402c890 con 0x7fd06c155270 2024-09-10T22:35:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.803+0000 7fd05e7fc640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd050077f70 0x7fd05007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.803+0000 7fd05e7fc640 1 -- 172.21.15.66:0/2165713903 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd0640ba3b0 con 0x7fd06c155270 2024-09-10T22:35:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.804+0000 7fd071c70640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd050077f70 0x7fd05007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.804+0000 7fd071c70640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd050077f70 0x7fd05007a430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd06c1eb6a0 tx=0x7fd068009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:17.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.808+0000 7fd05e7fc640 1 -- 172.21.15.66:0/2165713903 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd064032030 con 0x7fd06c155270 2024-09-10T22:35:17.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.994+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd06c0ce4e0 con 0x7fd06c155270 2024-09-10T22:35:17.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.995+0000 7fd05e7fc640 1 -- 172.21.15.66:0/2165713903 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16641 (secure 0 0 0) 0x7fd064082f40 con 0x7fd06c155270 2024-09-10T22:35:17.997 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:17.997 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:04.052417+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:17.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd050077f70 msgr2=0x7fd05007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd050077f70 0x7fd05007a430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd06c1eb6a0 tx=0x7fd068009040 comp rx=0 tx=0).stop 2024-09-10T22:35:17.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c155270 msgr2=0x7fd06c148f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:17.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c155270 0x7fd06c148f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd064009d60 tx=0x7fd064003f10 comp rx=0 tx=0).stop 2024-09-10T22:35:17.999 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 shutdown_connections 2024-09-10T22:35:18.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd050077f70 0x7fd05007a430 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd06c155270 0x7fd06c148f20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd06c154e90 0x7fd06c0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 --2- 172.21.15.66:0/2165713903 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd06c0ceda0 0x7fd06c1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.997+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 >> 172.21.15.66:0/2165713903 conn(0x7fd06c0b16d0 msgr2=0x7fd06c0cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.998+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 shutdown_connections 2024-09-10T22:35:18.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:17.998+0000 7fd073ed4640 1 -- 172.21.15.66:0/2165713903 wait complete. 2024-09-10T22:35:18.011 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:18.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.116+0000 7f67e39a0640 1 -- 172.21.15.66:0/4177605957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67c8004d40 msgr2=0x7f67c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.116+0000 7f67e39a0640 1 --2- 172.21.15.66:0/4177605957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f67d4001dc0 tx=0x7f67c8005830 comp rx=0 tx=0).stop 2024-09-10T22:35:18.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.116+0000 7f67e1f3d640 1 -- 172.21.15.66:0/4177605957 reap_dead start 2024-09-10T22:35:18.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.116+0000 7f67e39a0640 1 -- 172.21.15.66:0/4177605957 shutdown_connections 2024-09-10T22:35:18.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.116+0000 7f67e39a0640 1 -- 172.21.15.66:0/4177605957 >> 172.21.15.66:0/4177605957 conn(0x7f67dc0cd6e0 msgr2=0x7f67dc0cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.116+0000 7f67e39a0640 1 -- 172.21.15.66:0/4177605957 shutdown_connections 2024-09-10T22:35:18.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.116+0000 7f67e39a0640 1 -- 172.21.15.66:0/4177605957 wait complete. 2024-09-10T22:35:18.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.117+0000 7f67e39a0640 1 Processor -- start 2024-09-10T22:35:18.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.117+0000 7f67e39a0640 1 -- start start 2024-09-10T22:35:18.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e39a0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67c8004d40 0x7f67dc14f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e39a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67dc14bf80 0x7f67dc0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e39a0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67dc1f48a0 0x7f67dc1f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e39a0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc0b2ee0 con 0x7f67dc1f48a0 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e39a0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc0b2d60 con 0x7f67dc14bf80 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e39a0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f67dc06f830 con 0x7f67c8004d40 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e1f3d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67dc1f48a0 0x7f67dc1f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e1f3d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67dc1f48a0 0x7f67dc1f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36024/0 (socket says 172.21.15.66:36024) 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e1f3d640 1 -- 172.21.15.66:0/1833173369 learned_addr learned my addr 172.21.15.66:0/1833173369 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e173c640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67c8004d40 0x7f67dc14f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.118+0000 7f67e0f3b640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67dc14bf80 0x7f67dc0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e1f3d640 1 -- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67c8004d40 msgr2=0x7f67dc14f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e1f3d640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67c8004d40 0x7f67dc14f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e1f3d640 1 -- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67dc14bf80 msgr2=0x7f67dc0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e1f3d640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67dc14bf80 0x7f67dc0b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e1f3d640 1 -- 172.21.15.66:0/1833173369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc0b5250 con 0x7f67dc1f48a0 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e1f3d640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67dc1f48a0 0x7f67dc1f4d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67d402acc0 tx=0x7f67d4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67d1ffb640 1 -- 172.21.15.66:0/1833173369 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40060b0 con 0x7f67dc1f48a0 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67d1ffb640 1 -- 172.21.15.66:0/1833173369 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67d4006250 con 0x7f67dc1f48a0 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc0b5530 con 0x7f67dc1f48a0 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67d1ffb640 1 -- 172.21.15.66:0/1833173369 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40066a0 con 0x7f67dc1f48a0 2024-09-10T22:35:18.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.119+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc0b5af0 con 0x7f67dc1f48a0 2024-09-10T22:35:18.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.120+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc076550 con 0x7f67dc1f48a0 2024-09-10T22:35:18.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.121+0000 7f67d1ffb640 1 -- 172.21.15.66:0/1833173369 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f67d4033480 con 0x7f67dc1f48a0 2024-09-10T22:35:18.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.121+0000 7f67d1ffb640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f67c0077fa0 0x7f67c007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.121+0000 7f67d1ffb640 1 -- 172.21.15.66:0/1833173369 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67d40b9780 con 0x7f67dc1f48a0 2024-09-10T22:35:18.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.121+0000 7f67e173c640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f67c0077fa0 0x7f67c007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.122+0000 7f67e173c640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f67c0077fa0 0x7f67c007a460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67d8001410 tx=0x7f67d8006910 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.125+0000 7f67d1ffb640 1 -- 172.21.15.66:0/1833173369 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f67d40821f0 con 0x7f67dc1f48a0 2024-09-10T22:35:18.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.347+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67dc076550 con 0x7f67dc1f48a0 2024-09-10T22:35:18.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.347+0000 7f67d1ffb640 1 -- 172.21.15.66:0/1833173369 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f67d4081af0 con 0x7f67dc1f48a0 2024-09-10T22:35:18.349 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:18.349 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:18.349 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f67c0077fa0 msgr2=0x7f67c007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f67c0077fa0 0x7f67c007a460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67d8001410 tx=0x7f67d8006910 comp rx=0 tx=0).stop 2024-09-10T22:35:18.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67dc1f48a0 msgr2=0x7f67dc1f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67dc1f48a0 0x7f67dc1f4d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67d402acc0 tx=0x7f67d4001b10 comp rx=0 tx=0).stop 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 shutdown_connections 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f67c0077fa0 0x7f67c007a460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f67dc1f48a0 0x7f67dc1f4d50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67dc14bf80 0x7f67dc0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 --2- 172.21.15.66:0/1833173369 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f67c8004d40 0x7f67dc14f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 >> 172.21.15.66:0/1833173369 conn(0x7f67dc0cd6e0 msgr2=0x7f67dc0ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 shutdown_connections 2024-09-10T22:35:18.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.350+0000 7f67e39a0640 1 -- 172.21.15.66:0/1833173369 wait complete. 2024-09-10T22:35:18.765 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:35:02.418856+0000', 'modified': '2024-09-10T22:35:05.440441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4380}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4380': {'gid': 4380, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6834/695657263', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 695657263}, {'type': 'v1', 'addr': '172.21.15.66:6834', 'nonce': 695657263}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4380, 'qdb_cluster': [4380]} 2024-09-10T22:35:18.765 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:35:18.765 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:35:18.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.874+0000 7f9f1162a640 1 -- 172.21.15.66:0/3686427745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c0738a0 msgr2=0x7f9f0c070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.874+0000 7f9f1162a640 1 --2- 172.21.15.66:0/3686427745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c0738a0 0x7f9f0c070d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f04005fd0 tx=0x7f9f0402ab40 comp rx=0 tx=0).stop 2024-09-10T22:35:18.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.874+0000 7f9f1162a640 1 -- 172.21.15.66:0/3686427745 shutdown_connections 2024-09-10T22:35:18.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.874+0000 7f9f1162a640 1 --2- 172.21.15.66:0/3686427745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c0738a0 0x7f9f0c070d70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.875 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.874+0000 7f9f1162a640 1 -- 172.21.15.66:0/3686427745 >> 172.21.15.66:0/3686427745 conn(0x7f9f0c0b2750 msgr2=0x7f9f0c0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.874+0000 7f9f1162a640 1 -- 172.21.15.66:0/3686427745 shutdown_connections 2024-09-10T22:35:18.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.874+0000 7f9f1162a640 1 -- 172.21.15.66:0/3686427745 wait complete. 2024-09-10T22:35:18.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.875+0000 7f9f1162a640 1 Processor -- start 2024-09-10T22:35:18.876 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.875+0000 7f9f1162a640 1 -- start start 2024-09-10T22:35:18.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f1162a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0c06d230 0x7f9f0c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f1162a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f0c0738a0 0x7f9f0c15bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f1162a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c15dfd0 0x7f9f0c1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f1162a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c0af3d0 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f1162a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c0af250 con 0x7f9f0c06d230 2024-09-10T22:35:18.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f1162a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0c0af550 con 0x7f9f0c0738a0 2024-09-10T22:35:18.878 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c15dfd0 0x7f9f0c1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c15dfd0 0x7f9f0c1f7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36046/0 (socket says 172.21.15.66:36046) 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/1546612442 learned_addr learned my addr 172.21.15.66:0/1546612442 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0a7fc640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f0c0738a0 0x7f9f0c15bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0affd640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0c06d230 0x7f9f0c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f0c0738a0 msgr2=0x7f9f0c15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f0c0738a0 0x7f9f0c15bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0c06d230 msgr2=0x7f9f0c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0c06d230 0x7f9f0c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/1546612442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c0c77d0 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.876+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c15dfd0 0x7f9f0c1f7cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f04006010 tx=0x7f9f0402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.877+0000 7f9ef37fe640 1 -- 172.21.15.66:0/1546612442 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04044070 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.877+0000 7f9ef37fe640 1 -- 172.21.15.66:0/1546612442 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f04004250 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.877+0000 7f9ef37fe640 1 -- 172.21.15.66:0/1546612442 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f040046a0 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.879 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.877+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c15a6c0 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.880 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.877+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c15ac80 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.879+0000 7f9ef37fe640 1 -- 172.21.15.66:0/1546612442 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9f0402c9a0 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.879+0000 7f9ef37fe640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9ef8077f70 0x7f9ef807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.879+0000 7f9ef37fe640 1 -- 172.21.15.66:0/1546612442 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9f040ba690 con 0x7f9f0c15dfd0 2024-09-10T22:35:18.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.879+0000 7f9f0affd640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9ef8077f70 0x7f9ef807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.880+0000 7f9f143f6740 1 Processor -- start 2024-09-10T22:35:18.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.880+0000 7f9f0affd640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9ef8077f70 0x7f9ef807a430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9efc001720 tx=0x7f9efc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.881 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.880+0000 7f9f143f6740 1 -- start start 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbcf1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd3a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbcef5f0 0x55ffcbcefa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f143f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ffcbc66820 con 0x55ffcbcef5f0 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ffcbc55c50 con 0x55ffcbd1a1b0 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ffcbc59a60 con 0x55ffcbd39df0 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbcef5f0 0x55ffcbcefa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbcef5f0 0x55ffcbcefa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36048/0 (socket says 172.21.15.66:36048) 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/569180351 learned_addr learned my addr 172.21.15.66:0/569180351 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:18.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0a7fc640 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd3a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0affd640 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbcf1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 msgr2=0x55ffcbd3a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd3a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 msgr2=0x55ffcbcf1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbcf1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/569180351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcb9b7850 con 0x55ffcbcef5f0 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0affd640 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbcf1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbcef5f0 0x55ffcbcefa40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009600 tx=0x7f9f04082370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3650aa72fbf5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9ef0ff9640 1 -- 172.21.15.66:0/569180351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04082b40 con 0x55ffcbcef5f0 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9ef0ff9640 1 -- 172.21.15.66:0/569180351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f04087570 con 0x55ffcbcef5f0 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.881+0000 7f9ef0ff9640 1 -- 172.21.15.66:0/569180351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0408bf90 con 0x55ffcbcef5f0 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 -- 172.21.15.66:0/569180351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbcef5f0 msgr2=0x55ffcbcefa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbcef5f0 0x55ffcbcefa40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009600 tx=0x7f9f04082370 comp rx=0 tx=0).stop 2024-09-10T22:35:18.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 -- 172.21.15.66:0/569180351 shutdown_connections 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbcef5f0 0x55ffcbcefa40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd3a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 --2- 172.21.15.66:0/569180351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbcf1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 -- 172.21.15.66:0/569180351 >> 172.21.15.66:0/569180351 conn(0x55ffcbcf06a0 msgr2=0x55ffcbcf40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 -- 172.21.15.66:0/569180351 shutdown_connections 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.882+0000 7f9f143f6740 1 -- 172.21.15.66:0/569180351 wait complete. 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.883+0000 7f9f143f6740 1 -- start start 2024-09-10T22:35:18.886 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.883+0000 7f9f143f6740 1 Processor -- start 2024-09-10T22:35:18.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbd331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd11360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd30b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ffcb9c6620 con 0x55ffcbd12630 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ffcbbfb5f0 con 0x55ffcbd1a1b0 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ffcbcee850 con 0x55ffcbd39df0 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 10 client.0 _ll_register_callbacks cb 0x55ffcbbff0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd30b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd30b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36058/0 (socket says 172.21.15.66:36058) 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/2084788241 learned_addr learned my addr 172.21.15.66:0/2084788241 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0affd640 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbd331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0a7fc640 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd11360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 msgr2=0x55ffcbd11360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd11360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 msgr2=0x55ffcbd331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbd331d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/2084788241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcba2e6b0 con 0x55ffcbd12630 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd30b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f04082840 tx=0x7f9f040873b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15dbd8d1d1aa478 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/2084788241 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04082b40 con 0x55ffcbd12630 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/2084788241 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9f040a4bd0 con 0x55ffcbd12630 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.884+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f04087570 con 0x55ffcbd12630 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/2084788241 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f9f0406e0d0 con 0x55ffcbd12630 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 10 client.4722 fetch_fsmap learned FSMap version 8 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ffcbd4cc90 con 0x55ffcbd12630 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/2084788241 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f04087570 con 0x55ffcbd12630 2024-09-10T22:35:18.891 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/2084788241 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f9f04081da0 con 0x55ffcbd12630 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 10 client.4722 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 10 client.4722 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 20 client.4722 populate_metadata read hostname 'smithi066' 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbd31fb0 0x55ffcbd32390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 4 client.4722 mds_command: new command op to 4380 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 --> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffcbc68770 con 0x55ffcbd31fb0 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f0affd640 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbd31fb0 0x55ffcbd32390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9ee9ffb640 10 client.4722 ms_handle_connect on v2:172.21.15.66:6832/695657263 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.885+0000 7f9f0affd640 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbd31fb0 0x55ffcbd32390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb906019b8ec352a server_cookie=aa4075aa32dabcf7 in_seq=0 out_seq=0 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/2084788241 <== mds.0 v2:172.21.15.66:6832/695657263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffcbc68770 con 0x55ffcbd31fb0 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9ee9ffb640 10 client.4722 handle_command_reply: tid=0 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 client.4722 shutdown 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 msgr2=0x55ffcbd30b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.892 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd30b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f04082840 tx=0x7f9f040873b0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 shutdown_connections 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbd31fb0 0x55ffcbd32390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd30b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd39df0 0x55ffcbd11360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 --2- 172.21.15.66:0/2084788241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbd331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9ee9ffb640 0 client.4722 ms_handle_reset on v2:172.21.15.66:6832/695657263 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 >> 172.21.15.66:0/2084788241 conn(0x55ffcbcef5f0 msgr2=0x55ffcbced5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 shutdown_connections 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 1 -- 172.21.15.66:0/2084788241 wait complete. 2024-09-10T22:35:18.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.886+0000 7f9f143f6740 20 client.4722 trim_cache size 0 max 16384 2024-09-10T22:35:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 Processor -- start 2024-09-10T22:35:18.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 -- start start 2024-09-10T22:35:18.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd5c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd1a1b0 0x55ffcbd38ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd39df0 0x55ffcbd3a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c200300 con 0x55ffcbd12630 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c0c5450 con 0x55ffcbd39df0 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0c0c5aa0 con 0x55ffcbd1a1b0 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f0affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd5c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f0affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd5c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36074/0 (socket says 172.21.15.66:36074) 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.905+0000 7f9f0affd640 1 -- 172.21.15.66:0/3769772469 learned_addr learned my addr 172.21.15.66:0/3769772469 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f0affd640 1 -- 172.21.15.66:0/3769772469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd1a1b0 msgr2=0x55ffcbd38ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f0affd640 1 --2- 172.21.15.66:0/3769772469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd1a1b0 0x55ffcbd38ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f0affd640 1 -- 172.21.15.66:0/3769772469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd39df0 msgr2=0x55ffcbd3a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f0affd640 1 --2- 172.21.15.66:0/3769772469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd39df0 0x55ffcbd3a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f0affd640 1 -- 172.21.15.66:0/3769772469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcbce9160 con 0x55ffcbd12630 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f0affd640 1 --2- 172.21.15.66:0/3769772469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd5c950 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00fc00 tx=0x7f9efc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3c74590f04f361 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9eebfff640 1 -- 172.21.15.66:0/3769772469 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0249e0 con 0x55ffcbd12630 2024-09-10T22:35:18.909 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9eebfff640 1 -- 172.21.15.66:0/3769772469 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9efc02f070 con 0x55ffcbd12630 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9eebfff640 1 -- 172.21.15.66:0/3769772469 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc024e00 con 0x55ffcbd12630 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f143f6740 1 -- 172.21.15.66:0/3769772469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 msgr2=0x55ffcbd5c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.906+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3769772469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd5c950 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00fc00 tx=0x7f9efc012000 comp rx=0 tx=0).stop 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 -- 172.21.15.66:0/3769772469 shutdown_connections 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3769772469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd39df0 0x55ffcbd3a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3769772469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd1a1b0 0x55ffcbd38ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3769772469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd12630 0x55ffcbd5c950 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 -- 172.21.15.66:0/3769772469 >> 172.21.15.66:0/3769772469 conn(0x55ffcbcef5f0 msgr2=0x55ffcbd1ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 -- 172.21.15.66:0/3769772469 shutdown_connections 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 -- 172.21.15.66:0/3769772469 wait complete. 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 -- start start 2024-09-10T22:35:18.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.907+0000 7f9f143f6740 1 Processor -- start 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd12630 0x55ffcbd5c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbdbc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f143f6740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd39df0 0x55ffcbd38ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f143f6740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55ffcb9c6620 con 0x55ffcbd39df0 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55ffcbcee850 con 0x55ffcbd1a1b0 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f143f6740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55ffcbbfb5f0 con 0x55ffcbd12630 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f143f6740 10 client.0 _ll_register_callbacks cb 0x7f9f000012c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd39df0 0x55ffcbd38ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd39df0 0x55ffcbd38ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36078/0 (socket says 172.21.15.66:36078) 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/3809410394 learned_addr learned my addr 172.21.15.66:0/3809410394 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0affd640 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd12630 0x55ffcbd5c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0a7fc640 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbdbc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd12630 msgr2=0x55ffcbd5c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd12630 0x55ffcbd5c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 msgr2=0x55ffcbdbc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbdbc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.908+0000 7f9f0b7fe640 1 -- 172.21.15.66:0/3809410394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcbd4f110 con 0x55ffcbd39df0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f0b7fe640 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd39df0 0x55ffcbd38ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f0407fdb0 tx=0x7f9f040b8be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5564b8f1b9c2aa25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/3809410394 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04082490 con 0x55ffcbd39df0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/3809410394 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9f0406bb30 con 0x55ffcbd39df0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9efc02f070 con 0x55ffcbd39df0 2024-09-10T22:35:18.915 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/3809410394 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f9f04087330 con 0x55ffcbd39df0 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 10 client.4728 fetch_fsmap learned FSMap version 8 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ffcbd4f110 con 0x55ffcbd39df0 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/3809410394 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9efc02f070 con 0x55ffcbd39df0 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/3809410394 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f9f04081810 con 0x55ffcbd39df0 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 10 client.4728 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 10 client.4728 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 20 client.4728 populate_metadata read hostname 'smithi066' 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbdbd980 0x55ffcbdbdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 4 client.4728 mds_command: new command op to 4380 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 --> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] -- command(tid 0: {"prefix": "status"}) -- 0x55ffcbc64220 con 0x55ffcbdbd980 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.909+0000 7f9f0affd640 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbdbd980 0x55ffcbdbdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9ee9ffb640 10 client.4728 ms_handle_connect on v2:172.21.15.66:6832/695657263 2024-09-10T22:35:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f0affd640 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbdbd980 0x55ffcbdbdd60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9971a564827ea57d server_cookie=a1dd839e246498e5 in_seq=0 out_seq=0 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9ee9ffb640 1 -- 172.21.15.66:0/3809410394 <== mds.0 v2:172.21.15.66:6832/695657263 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55ffcbc64220 con 0x55ffcbdbd980 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9ee9ffb640 10 client.4728 handle_command_reply: tid=0 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 client.4728 shutdown 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd39df0 msgr2=0x55ffcbd38ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd39df0 0x55ffcbd38ca0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f0407fdb0 tx=0x7f9f040b8be0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 shutdown_connections 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:6832/695657263,v1:172.21.15.66:6834/695657263] conn(0x55ffcbdbd980 0x55ffcbdbdd60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ffcbd39df0 0x55ffcbd38ca0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffcbd1a1b0 0x55ffcbdbc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 --2- 172.21.15.66:0/3809410394 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55ffcbd12630 0x55ffcbd5c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 >> 172.21.15.66:0/3809410394 conn(0x55ffcbcef5f0 msgr2=0x55ffcbd1ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9ee9ffb640 0 client.4728 ms_handle_reset on v2:172.21.15.66:6832/695657263 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 shutdown_connections 2024-09-10T22:35:18.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 1 -- 172.21.15.66:0/3809410394 wait complete. 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f143f6740 20 client.4728 trim_cache size 0 max 16384 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "7ef137cd-a1ce-4819-b27d-b71268612f85", 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4380, 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 15.857761088, 2024-09-10T22:35:18.918 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 8, 2024-09-10T22:35:18.919 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 21, 2024-09-10T22:35:18.919 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 21, 2024-09-10T22:35:18.919 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 21.894819669 2024-09-10T22:35:18.919 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:35:18.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9ef8077f70 msgr2=0x7f9ef807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f1162a640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9ef8077f70 0x7f9ef807a430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9efc001720 tx=0x7f9efc009040 comp rx=0 tx=0).stop 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c15dfd0 msgr2=0x7f9f0c1f7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.910+0000 7f9f1162a640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c15dfd0 0x7f9f0c1f7cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f04006010 tx=0x7f9f0402c8b0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 shutdown_connections 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9ef8077f70 0x7f9ef807a430 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f0c15dfd0 0x7f9f0c1f7cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9f0c0738a0 0x7f9f0c15bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 --2- 172.21.15.66:0/1546612442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f0c06d230 0x7f9f0c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 >> 172.21.15.66:0/1546612442 conn(0x7f9f0c0b2750 msgr2=0x7f9f0c06cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 shutdown_connections 2024-09-10T22:35:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:18.911+0000 7f9f1162a640 1 -- 172.21.15.66:0/1546612442 wait complete. 2024-09-10T22:35:18.922 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T22:35:18.950 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T22:35:18.950 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T22:35:18.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:18.951 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:35:18.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:18.969 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-09-10T22:35:19.035 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-09-10T22:35:19.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:19.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.037 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:35:19.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.093 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-09-10T22:35:19.160 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-09-10T22:35:19.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:19.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.162 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:35:19.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T22:35:19.252 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-09-10T22:35:19.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:19.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.254 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:35:19.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.312 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T22:35:19.380 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-09-10T22:35:19.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:35:19.382 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-10T22:35:19.382 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:35:19.382 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:19.382 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:35:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.486+0000 7ff28dab0640 1 -- 172.21.15.66:0/4018553712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 msgr2=0x7ff270004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.486+0000 7ff28dab0640 1 --2- 172.21.15.66:0/4018553712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 0x7ff270004060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff278001830 tx=0x7ff270005830 comp rx=0 tx=0).stop 2024-09-10T22:35:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.486+0000 7ff287fff640 1 -- 172.21.15.66:0/4018553712 reap_dead start 2024-09-10T22:35:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.486+0000 7ff28dab0640 1 -- 172.21.15.66:0/4018553712 shutdown_connections 2024-09-10T22:35:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.486+0000 7ff28dab0640 1 -- 172.21.15.66:0/4018553712 >> 172.21.15.66:0/4018553712 conn(0x7ff2880b16d0 msgr2=0x7ff2880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.487+0000 7ff28dab0640 1 -- 172.21.15.66:0/4018553712 shutdown_connections 2024-09-10T22:35:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.487+0000 7ff28dab0640 1 -- 172.21.15.66:0/4018553712 wait complete. 2024-09-10T22:35:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.488+0000 7ff28dab0640 1 Processor -- start 2024-09-10T22:35:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.488+0000 7ff28dab0640 1 -- start start 2024-09-10T22:35:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff28dab0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 0x7ff2881439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff28dab0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff28814ca00 0x7ff2881f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff28dab0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff28814ce80 0x7ff2881f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff28dab0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880cebf0 con 0x7ff270003c60 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff28dab0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880cea70 con 0x7ff28814ca00 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff28dab0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff2880731e0 con 0x7ff28814ce80 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 0x7ff2881439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 0x7ff2881439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36100/0 (socket says 172.21.15.66:36100) 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 -- 172.21.15.66:0/529446846 learned_addr learned my addr 172.21.15.66:0/529446846 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff286ffd640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff28814ca00 0x7ff2881f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff287fff640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff28814ce80 0x7ff2881f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 -- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff28814ce80 msgr2=0x7ff2881f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff28814ce80 0x7ff2881f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 -- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff28814ca00 msgr2=0x7ff2881f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff28814ca00 0x7ff2881f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 -- 172.21.15.66:0/529446846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff288142f80 con 0x7ff270003c60 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff2877fe640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 0x7ff2881439f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff278000f60 tx=0x7ff2780060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff26ffff640 1 -- 172.21.15.66:0/529446846 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278043070 con 0x7ff270003c60 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff26ffff640 1 -- 172.21.15.66:0/529446846 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff278003af0 con 0x7ff270003c60 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff26ffff640 1 -- 172.21.15.66:0/529446846 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780064c0 con 0x7ff270003c60 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.489+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881f67f0 con 0x7ff270003c60 2024-09-10T22:35:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.490+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2881434d0 con 0x7ff270003c60 2024-09-10T22:35:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.491+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff288072ad0 con 0x7ff270003c60 2024-09-10T22:35:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.491+0000 7ff26ffff640 1 -- 172.21.15.66:0/529446846 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff278033480 con 0x7ff270003c60 2024-09-10T22:35:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.492+0000 7ff26ffff640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff274077f70 0x7ff27407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.492+0000 7ff26ffff640 1 -- 172.21.15.66:0/529446846 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2780b9d30 con 0x7ff270003c60 2024-09-10T22:35:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.492+0000 7ff286ffd640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff274077f70 0x7ff27407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.492+0000 7ff286ffd640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff274077f70 0x7ff27407a430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff28814be80 tx=0x7ff27c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.496+0000 7ff26ffff640 1 -- 172.21.15.66:0/529446846 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff2780822a0 con 0x7ff270003c60 2024-09-10T22:35:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.718+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff288075ad0 con 0x7ff270003c60 2024-09-10T22:35:19.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.718+0000 7ff26ffff640 1 -- 172.21.15.66:0/529446846 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ff278086cd0 con 0x7ff270003c60 2024-09-10T22:35:19.720 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff274077f70 msgr2=0x7ff27407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff274077f70 0x7ff27407a430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff28814be80 tx=0x7ff27c005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 msgr2=0x7ff2881439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 0x7ff2881439f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff278000f60 tx=0x7ff2780060e0 comp rx=0 tx=0).stop 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 shutdown_connections 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff274077f70 0x7ff27407a430 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff28814ce80 0x7ff2881f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff28814ca00 0x7ff2881f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:19.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 --2- 172.21.15.66:0/529446846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff270003c60 0x7ff2881439f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.720+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 >> 172.21.15.66:0/529446846 conn(0x7ff2880b16d0 msgr2=0x7ff288153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.721+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 shutdown_connections 2024-09-10T22:35:19.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19.721+0000 7ff28dab0640 1 -- 172.21.15.66:0/529446846 wait complete. 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:35:19.733 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:35:19.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.734 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:19.750 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42778sec preferred_lft 42778sec 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link noprefixroute 2024-09-10T22:35:19.751 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:19.751 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:35:19.752 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:19.752 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:19.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:35:19.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2024-09-10T22:35:19.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2024-09-10T22:35:19.752 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:35:19.752 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:19.838 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:19.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:19.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.959 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:35:19.985 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:35:19.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:19.987 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:35:20.072 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:35:20.072 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:35:20.072 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:35:20.072 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:35:20.072 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:35:20.073 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:20.073 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:20.073 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:35:20.073 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:35:20.073 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:35:20.073 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:20.314 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:20.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:20.321 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:35:20.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:20.337 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-09-10T22:35:20.395 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:20.395 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:20.395 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:35:20.396 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T22:35:20.396 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:20.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:20.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:20.519 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T22:35:20.519 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:20.519 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:20.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T22:35:20.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:35:20.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T22:35:20.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T22:35:20.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T22:35:20.520 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:20.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:20.653 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:20.653 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:20.653 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:35:20.653 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:35:20.653 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:20.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:20.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:20.742 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:20.770 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:35:20.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:20.771 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:20.826 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T22:35:20.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:20.827 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:20.923 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:20.923 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:21.060 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:21.060 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:21.060 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.94:3300,172.21.15.94:3301" 2024-09-10T22:35:21.060 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with new device syntax: 0@7ef137cd-a1ce-4819-b27d-b71268612f85.cephfs=/ 2024-09-10T22:35:21.060 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.66:3300/172.21.15.94:3300/172.21.15.94:3301" will pass to kernel 2024-09-10T22:35:21.061 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T22:35:21.061 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-09-10T22:35:21.061 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T22:35:21.061 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T22:35:21.061 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T22:35:21.063 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:35:21.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:21.063 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:21.090 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:21.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:21.195 INFO:teuthology.orchestra.run.smithi066.stdout:client4681 2024-09-10T22:35:21.195 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:21.196 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:21.212 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:21.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:21.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:21.315 INFO:teuthology.orchestra.run.smithi066.stdout:client4681 2024-09-10T22:35:21.315 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:21.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:21.377 INFO:teuthology.orchestra.run.smithi066.stdout:client4681 2024-09-10T22:35:21.377 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ef137cd-a1ce-4819-b27d-b71268612f85.client4681/status') 2024-09-10T22:35:21.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:21.440 INFO:teuthology.orchestra.run.smithi066.stdout:instance: client.4681 (3)172.21.15.66:0/1951713351 2024-09-10T22:35:21.441 INFO:teuthology.orchestra.run.smithi066.stdout:blocklisted: false 2024-09-10T22:35:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records in 2024-09-10T22:35:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records out 2024-09-10T22:35:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:70 bytes copied, 2.4351e-05 s, 2.9 MB/s 2024-09-10T22:35:21.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:21.448 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-10T22:35:21.448 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:35:21.448 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T22:35:21.448 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 -- 172.21.15.66:0/2779054722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef300d1ae0 msgr2=0x7fef300b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 --2- 172.21.15.66:0/2779054722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef300d1ae0 0x7fef300b4450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef300a1e30 tx=0x7fef2002a8b0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 -- 172.21.15.66:0/2779054722 shutdown_connections 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 --2- 172.21.15.66:0/2779054722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef300d1ae0 0x7fef300b4450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 --2- 172.21.15.66:0/2779054722 >> v2:172.21.15.94:3300/0 conn(0x7fef300d2be0 0x7fef300d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 -- 172.21.15.66:0/2779054722 >> 172.21.15.66:0/2779054722 conn(0x7fef300d3f10 msgr2=0x7fef300d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 -- 172.21.15.66:0/2779054722 shutdown_connections 2024-09-10T22:35:21.553 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.551+0000 7fef38447640 1 -- 172.21.15.66:0/2779054722 wait complete. 2024-09-10T22:35:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.553+0000 7fef38447640 1 Processor -- start 2024-09-10T22:35:21.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.553+0000 7fef38447640 1 -- start start 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.553+0000 7fef38447640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef300d1ae0 0x7fef301f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.553+0000 7fef38447640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef300d2be0 0x7fef3009ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef38447640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef3015e090 0x7fef3015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef38447640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fef300b4d20 con 0x7fef3015e090 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef38447640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fef300b4ba0 con 0x7fef300d1ae0 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef38447640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fef300b4ea0 con 0x7fef300d2be0 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef3015e090 0x7fef3015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef3015e090 0x7fef3015c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36122/0 (socket says 172.21.15.66:36122) 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 -- 172.21.15.66:0/4287975789 learned_addr learned my addr 172.21.15.66:0/4287975789 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef359e2640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef300d2be0 0x7fef3009ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:21.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef361e3640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef300d1ae0 0x7fef301f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 -- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef300d2be0 msgr2=0x7fef3009ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef300d2be0 0x7fef3009ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 -- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef300d1ae0 msgr2=0x7fef301f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef300d1ae0 0x7fef301f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 -- 172.21.15.66:0/4287975789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef300cdbf0 con 0x7fef3015e090 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef361e3640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef300d1ae0 0x7fef301f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef369e4640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef3015e090 0x7fef3015c3a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef20009ca0 tx=0x7fef20031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef2effd640 1 -- 172.21.15.66:0/4287975789 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20040070 con 0x7fef3015e090 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef2effd640 1 -- 172.21.15.66:0/4287975789 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef20004040 con 0x7fef3015e090 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef300cded0 con 0x7fef3015e090 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef2effd640 1 -- 172.21.15.66:0/4287975789 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20004470 con 0x7fef3015e090 2024-09-10T22:35:21.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.554+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef300cb360 con 0x7fef3015e090 2024-09-10T22:35:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.555+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef300b4450 con 0x7fef3015e090 2024-09-10T22:35:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.556+0000 7fef2effd640 1 -- 172.21.15.66:0/4287975789 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fef20004610 con 0x7fef3015e090 2024-09-10T22:35:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.556+0000 7fef2effd640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fef1c077f70 0x7fef1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.557+0000 7fef2effd640 1 -- 172.21.15.66:0/4287975789 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fef200bb630 con 0x7fef3015e090 2024-09-10T22:35:21.558 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.557+0000 7fef361e3640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fef1c077f70 0x7fef1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:21.559 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.557+0000 7fef361e3640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fef1c077f70 0x7fef1c07a430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef300b38a0 tx=0x7fef28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:21.561 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.561+0000 7fef2effd640 1 -- 172.21.15.66:0/4287975789 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fef200841c0 con 0x7fef3015e090 2024-09-10T22:35:21.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.785+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fef300d4670 con 0x7fef3015e090 2024-09-10T22:35:21.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.785+0000 7fef2effd640 1 -- 172.21.15.66:0/4287975789 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fef20083ac0 con 0x7fef3015e090 2024-09-10T22:35:21.787 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fef1c077f70 msgr2=0x7fef1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fef1c077f70 0x7fef1c07a430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef300b38a0 tx=0x7fef28009040 comp rx=0 tx=0).stop 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef3015e090 msgr2=0x7fef3015c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef3015e090 0x7fef3015c3a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef20009ca0 tx=0x7fef20031720 comp rx=0 tx=0).stop 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 shutdown_connections 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fef1c077f70 0x7fef1c07a430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fef3015e090 0x7fef3015c3a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fef300d2be0 0x7fef3009ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 --2- 172.21.15.66:0/4287975789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef300d1ae0 0x7fef301f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 >> 172.21.15.66:0/4287975789 conn(0x7fef300d3f10 msgr2=0x7fef300b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:21.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 shutdown_connections 2024-09-10T22:35:21.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:21.788+0000 7fef38447640 1 -- 172.21.15.66:0/4287975789 wait complete. 2024-09-10T22:35:21.799 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:35:21.799 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:35:21.799 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:35:21.799 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T22:35:21.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T22:35:21.800 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:35:21.800 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:35:21.800 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:35:21.800 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T22:35:21.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:21.800 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:35:21.816 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:21.816 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:21.817 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42776sec preferred_lft 42776sec 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link noprefixroute 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether f2:19:d9:d3:e1:f4 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::f82d:d6ff:fec7:dfcb/64 scope link 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether f2:19:d9:d3:e1:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::f019:d9ff:fed3:e1f4/64 scope link tentative 2024-09-10T22:35:21.818 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:21.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:21.819 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:35:21.881 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:35:21.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:21.882 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:35:21.940 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:35:21.940 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:35:21.940 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:35:21.940 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:35:21.940 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:35:21.940 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:21.941 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:21.941 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:35:21.941 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:35:21.941 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:35:21.941 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:22.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:22.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:22.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:22.069 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:35:22.086 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:35:22.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:22.086 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2024-09-10T22:35:22.142 INFO:teuthology.orchestra.run.smithi066.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T22:35:22.143 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:22.143 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:22.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:35:22.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T22:35:22.143 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:22.232 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:22.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:22.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:22.261 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T22:35:22.287 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:22.287 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:22.287 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:22.287 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.287 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:22.287 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.288 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:35:22.288 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c2:47:6a:65:ff:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:35:22.288 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:35:22.288 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.288 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c047:6aff:fe65:ff3e/64 scope link 2024-09-10T22:35:22.288 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:22.289 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T22:35:22.354 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:22.354 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:22.354 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether c2:47:6a:65:ff:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::c047:6aff:fe65:ff3e/64 scope link 2024-09-10T22:35:22.355 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:22.356 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T22:35:22.356 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:22.357 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:22.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T22:35:22.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T22:35:22.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T22:35:22.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T22:35:22.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T22:35:22.357 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:22.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:22.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:22.513 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:22.513 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:22.513 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.1 up 2024-09-10T22:35:22.513 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T22:35:22.513 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:22.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:22.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:22.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:22.592 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T22:35:22.607 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T22:35:22.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:22.607 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T22:35:22.659 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-10T22:35:22.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:22.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:22.745 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:22.745 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:22.753 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:22.753 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:22.753 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.94:3300,172.21.15.94:3301" 2024-09-10T22:35:22.753 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with new device syntax: 2@7ef137cd-a1ce-4819-b27d-b71268612f85.cephfs=/ 2024-09-10T22:35:22.753 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.66:3300/172.21.15.94:3300/172.21.15.94:3301" will pass to kernel 2024-09-10T22:35:22.754 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-10T22:35:22.754 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-09-10T22:35:22.754 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T22:35:22.754 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T22:35:22.754 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T22:35:22.755 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:35:22.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:22.755 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T22:35:22.781 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:22.871 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:22.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:22.879 INFO:teuthology.orchestra.run.smithi066.stdout:client4505 2024-09-10T22:35:22.880 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:22.880 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:22.897 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:22.987 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:22.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:22.994 INFO:teuthology.orchestra.run.smithi066.stdout:client4505 2024-09-10T22:35:22.994 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:23.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:23.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:23.051 INFO:teuthology.orchestra.run.smithi066.stdout:client4505 2024-09-10T22:35:23.051 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ef137cd-a1ce-4819-b27d-b71268612f85.client4505/status') 2024-09-10T22:35:23.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:23.111 INFO:teuthology.orchestra.run.smithi066.stdout:instance: client.4505 (3)172.21.15.66:0/4260954851 2024-09-10T22:35:23.111 INFO:teuthology.orchestra.run.smithi066.stdout:blocklisted: false 2024-09-10T22:35:23.111 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records in 2024-09-10T22:35:23.112 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records out 2024-09-10T22:35:23.112 INFO:teuthology.orchestra.run.smithi066.stderr:70 bytes copied, 2.5122e-05 s, 2.8 MB/s 2024-09-10T22:35:23.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:23.119 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-10T22:35:23.119 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:35:23.119 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T22:35:23.119 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:35:23.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 -- 172.21.15.66:0/2891180273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415acc0 msgr2=0x7fdec40cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:23.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 --2- 172.21.15.66:0/2891180273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415acc0 0x7fdec40cff90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdec406eb90 tx=0x7fdebc02aac0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 -- 172.21.15.66:0/2891180273 shutdown_connections 2024-09-10T22:35:23.224 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 --2- 172.21.15.66:0/2891180273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415acc0 0x7fdec40cff90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 --2- 172.21.15.66:0/2891180273 >> v2:172.21.15.94:3300/0 conn(0x7fdec40cc1b0 0x7fdec40cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 --2- 172.21.15.66:0/2891180273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdec415a8e0 0x7fdec40ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 -- 172.21.15.66:0/2891180273 >> 172.21.15.66:0/2891180273 conn(0x7fdec40d0c90 msgr2=0x7fdec40d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:23.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 -- 172.21.15.66:0/2891180273 shutdown_connections 2024-09-10T22:35:23.225 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.222+0000 7fdec930e640 1 -- 172.21.15.66:0/2891180273 wait complete. 2024-09-10T22:35:23.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.223+0000 7fdec930e640 1 Processor -- start 2024-09-10T22:35:23.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.223+0000 7fdec930e640 1 -- start start 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec930e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415a8e0 0x7fdec414be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec930e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec415acc0 0x7fdec414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec930e640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdec40ac380 0x7fdec406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec930e640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4070a20 con 0x7fdec415a8e0 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec930e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdec40d0900 con 0x7fdec415acc0 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec930e640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdec4070ba0 con 0x7fdec40ac380 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415a8e0 0x7fdec414be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415a8e0 0x7fdec414be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36138/0 (socket says 172.21.15.66:36138) 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 -- 172.21.15.66:0/3611354182 learned_addr learned my addr 172.21.15.66:0/3611354182 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdebbfff640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec415acc0 0x7fdec414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec37fe640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdec40ac380 0x7fdec406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:23.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 -- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdec40ac380 msgr2=0x7fdec406cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdec40ac380 0x7fdec406cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 -- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec415acc0 msgr2=0x7fdec414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec415acc0 0x7fdec414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 -- 172.21.15.66:0/3611354182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec414cb20 con 0x7fdec415a8e0 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdebbfff640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec415acc0 0x7fdec414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec37fe640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdec40ac380 0x7fdec406cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdec2ffd640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415a8e0 0x7fdec414be20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d950 tx=0x7fdeb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.224+0000 7fdebb7fe640 1 -- 172.21.15.66:0/3611354182 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017030 con 0x7fdec415a8e0 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.225+0000 7fdebb7fe640 1 -- 172.21.15.66:0/3611354182 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdeb4009b20 con 0x7fdec415a8e0 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.225+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec41f2840 con 0x7fdec415a8e0 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.225+0000 7fdebb7fe640 1 -- 172.21.15.66:0/3611354182 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4004020 con 0x7fdec415a8e0 2024-09-10T22:35:23.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.225+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec41f2e00 con 0x7fdec415a8e0 2024-09-10T22:35:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.226+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec40ce860 con 0x7fdec415a8e0 2024-09-10T22:35:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.226+0000 7fdebb7fe640 1 -- 172.21.15.66:0/3611354182 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdeb40041c0 con 0x7fdec415a8e0 2024-09-10T22:35:23.230 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.227+0000 7fdebb7fe640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdea4077f70 0x7fdea407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.227+0000 7fdebb7fe640 1 -- 172.21.15.66:0/3611354182 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdeb4099e40 con 0x7fdec415a8e0 2024-09-10T22:35:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.227+0000 7fdebbfff640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdea4077f70 0x7fdea407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:23.231 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.228+0000 7fdebbfff640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdea4077f70 0x7fdea407a430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdec40ca5b0 tx=0x7fdeac005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:23.233 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.231+0000 7fdebb7fe640 1 -- 172.21.15.66:0/3611354182 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fdeb40629d0 con 0x7fdec415a8e0 2024-09-10T22:35:23.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.453+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdec4074720 con 0x7fdec415a8e0 2024-09-10T22:35:23.455 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.453+0000 7fdebb7fe640 1 -- 172.21.15.66:0/3611354182 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fdeb406b0f0 con 0x7fdec415a8e0 2024-09-10T22:35:23.456 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdea4077f70 msgr2=0x7fdea407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdea4077f70 0x7fdea407a430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdec40ca5b0 tx=0x7fdeac005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415a8e0 msgr2=0x7fdec414be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415a8e0 0x7fdec414be20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d950 tx=0x7fdeb400de10 comp rx=0 tx=0).stop 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 shutdown_connections 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdea4077f70 0x7fdea407a430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdec40ac380 0x7fdec406cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdec415acc0 0x7fdec414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 --2- 172.21.15.66:0/3611354182 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdec415a8e0 0x7fdec414be20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:23.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.455+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 >> 172.21.15.66:0/3611354182 conn(0x7fdec40d0c90 msgr2=0x7fdec415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:23.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.456+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 shutdown_connections 2024-09-10T22:35:23.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:23.456+0000 7fdec930e640 1 -- 172.21.15.66:0/3611354182 wait complete. 2024-09-10T22:35:23.468 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:35:23.468 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:35:23.468 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T22:35:23.468 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T22:35:23.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T22:35:23.469 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:35:23.469 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:35:23.469 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:35:23.469 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T22:35:23.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:23.469 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:23.489 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42739sec preferred_lft 42739sec 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link noprefixroute 2024-09-10T22:35:23.490 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:23.491 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:35:23.491 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:23.491 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:23.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:35:23.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-09-10T22:35:23.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-09-10T22:35:23.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:35:23.491 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:23.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:23.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:23.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:23.780 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:35:23.806 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T22:35:23.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:23.807 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T22:35:23.881 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T22:35:23.881 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:35:23.881 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:35:23.882 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:35:23.882 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:35:23.882 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:23.882 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:23.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:35:23.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:35:23.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:35:23.882 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:23.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:24.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:24.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:24.155 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:35:24.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:24.171 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-09-10T22:35:24.225 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:24.226 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:24.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:35:24.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T22:35:24.226 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:24.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:24.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:24.365 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T22:35:24.366 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:24.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:24.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:24.527 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:24.527 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:24.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-09-10T22:35:24.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:35:24.528 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:24.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:24.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:24.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:24.609 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T22:35:24.624 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T22:35:24.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:24.624 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T22:35:24.679 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T22:35:24.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:24.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:24.769 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:24.770 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.94:3300,172.21.15.94:3301" 2024-09-10T22:35:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with new device syntax: 1@7ef137cd-a1ce-4819-b27d-b71268612f85.cephfs=/ 2024-09-10T22:35:24.892 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.66:3300/172.21.15.94:3300/172.21.15.94:3301" will pass to kernel 2024-09-10T22:35:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-10T22:35:24.893 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-09-10T22:35:24.893 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T22:35:24.893 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T22:35:24.893 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T22:35:24.895 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:35:24.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:24.895 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T22:35:24.920 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:25.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:25.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:25.016 INFO:teuthology.orchestra.run.smithi094.stdout:client4752 2024-09-10T22:35:25.016 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:25.016 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:25.033 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:25.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:25.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:25.129 INFO:teuthology.orchestra.run.smithi094.stdout:client4752 2024-09-10T22:35:25.130 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:25.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:25.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:25.185 INFO:teuthology.orchestra.run.smithi094.stdout:client4752 2024-09-10T22:35:25.185 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ef137cd-a1ce-4819-b27d-b71268612f85.client4752/status') 2024-09-10T22:35:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:25.243 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.4752 (3)172.21.15.94:0/3708637070 2024-09-10T22:35:25.244 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-09-10T22:35:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-09-10T22:35:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-09-10T22:35:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:70 bytes copied, 2.2672e-05 s, 3.1 MB/s 2024-09-10T22:35:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:25.250 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-10T22:35:25.251 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:35:25.251 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T22:35:25.251 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:35:25.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/564398398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 msgr2=0x7f8c9c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:25.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/564398398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c0ceab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c08c290 tx=0x7f8c9402aa30 comp rx=0 tx=0).stop 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/564398398 shutdown_connections 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/564398398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c0ceab0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/564398398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c9c0d1ae0 0x7f8c9c0d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/564398398 >> v2:172.21.15.94:3300/0 conn(0x7f8c9c15e090 0x7f8c9c0d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/564398398 >> 172.21.15.66:0/564398398 conn(0x7f8c9c0d3f10 msgr2=0x7f8c9c0d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/564398398 shutdown_connections 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.367+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/564398398 wait complete. 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.368+0000 7f8ca0eb0640 1 Processor -- start 2024-09-10T22:35:25.370 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8ca0eb0640 1 -- start start 2024-09-10T22:35:25.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8ca0eb0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c9c0d1ae0 0x7f8c9c152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8ca0eb0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c1f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8ca0eb0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c9c15e090 0x7f8c9c0b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8ca0eb0640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c0cf380 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8ca0eb0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c0cf200 con 0x7f8c9c0d1ae0 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8ca0eb0640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9c0cf500 con 0x7f8c9c15e090 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8c99d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c1f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.369+0000 7f8c99d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c1f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46592/0 (socket says 172.21.15.66:46592) 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c99d9b640 1 -- 172.21.15.66:0/1741818355 learned_addr learned my addr 172.21.15.66:0/1741818355 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c9ad9d640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c9c15e090 0x7f8c9c0b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c9a59c640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c9c0d1ae0 0x7f8c9c152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c99d9b640 1 -- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c9c15e090 msgr2=0x7f8c9c0b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c99d9b640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c9c15e090 0x7f8c9c0b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c99d9b640 1 -- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c9c0d1ae0 msgr2=0x7f8c9c152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c99d9b640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c9c0d1ae0 0x7f8c9c152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c99d9b640 1 -- 172.21.15.66:0/1741818355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c1f8d50 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c9ad9d640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c9c15e090 0x7f8c9c0b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c99d9b640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c1f8810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e9f0 tx=0x7f8c9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c8affd640 1 -- 172.21.15.66:0/1741818355 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000cd80 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c8affd640 1 -- 172.21.15.66:0/1741818355 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c90004030 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8c8affd640 1 -- 172.21.15.66:0/1741818355 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90004480 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9c1584f0 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.370+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9c158ab0 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.371+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9c0d24d0 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.372+0000 7f8c8affd640 1 -- 172.21.15.66:0/1741818355 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8c900035f0 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.373+0000 7f8c8affd640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8c7c077f70 0x7f8c7c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:25.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.373+0000 7f8c8affd640 1 -- 172.21.15.66:0/1741818355 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8c9009a6d0 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.373+0000 7f8c9a59c640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8c7c077f70 0x7f8c7c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:25.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.373+0000 7f8c9a59c640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8c7c077f70 0x7f8c7c07a430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009ab0 tx=0x7f8c8c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:25.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.377+0000 7f8c8affd640 1 -- 172.21.15.66:0/1741818355 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8c9c1f8d50 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.599+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8c9c0b5f80 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.600+0000 7f8c8affd640 1 -- 172.21.15.66:0/1741818355 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8c90062c10 con 0x7f8c9c0d2a10 2024-09-10T22:35:25.602 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:35:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8c7c077f70 msgr2=0x7f8c7c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8c7c077f70 0x7f8c7c07a430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009ab0 tx=0x7f8c8c005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 msgr2=0x7f8c9c1f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c1f8810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000e9f0 tx=0x7f8c9000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 shutdown_connections 2024-09-10T22:35:25.604 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8c7c077f70 0x7f8c7c07a430 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8c9c15e090 0x7f8c9c0b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c9c0d2a10 0x7f8c9c1f8810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 --2- 172.21.15.66:0/1741818355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c9c0d1ae0 0x7f8c9c152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 >> 172.21.15.66:0/1741818355 conn(0x7f8c9c0d3f10 msgr2=0x7f8c9c0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 shutdown_connections 2024-09-10T22:35:25.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:25.602+0000 7f8ca0eb0640 1 -- 172.21.15.66:0/1741818355 wait complete. 2024-09-10T22:35:25.614 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:35:25.614 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:35:25.615 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-09-10T22:35:25.615 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T22:35:25.615 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T22:35:25.615 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:35:25.615 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:35:25.615 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:35:25.615 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T22:35:25.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:25.615 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-09-10T22:35:25.631 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link noprefixroute 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 36:75:f3:6b:e5:9e brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::e4f6:1fff:fe25:4172/64 scope link 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 36:75:f3:6b:e5:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::3475:f3ff:fe6b:e59e/64 scope link tentative 2024-09-10T22:35:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:25.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:25.634 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:35:25.695 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-09-10T22:35:25.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:25.697 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-09-10T22:35:25.754 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-09-10T22:35:25.754 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:35:25.754 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:35:25.754 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:35:25.754 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:35:25.754 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:25.755 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:25.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:35:25.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:35:25.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:35:25.755 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:25.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:25.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:25.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:25.880 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:35:25.896 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T22:35:25.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:25.897 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-09-10T22:35:25.952 INFO:teuthology.orchestra.run.smithi094.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T22:35:25.952 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:25.952 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:25.952 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:35:25.952 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T22:35:25.952 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:26.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:26.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:26.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:26.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T22:35:26.094 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.095 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:35:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether be:c8:73:c1:75:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:35:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:35:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::bcc8:73ff:fec1:7521/64 scope link tentative 2024-09-10T22:35:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:26.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T22:35:26.161 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether be:c8:73:c1:75:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::bcc8:73ff:fec1:7521/64 scope link tentative 2024-09-10T22:35:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:26.164 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T22:35:26.164 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:26.164 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:26.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T22:35:26.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T22:35:26.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T22:35:26.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T22:35:26.165 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T22:35:26.165 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:26.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:26.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:26.318 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:26.318 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-09-10T22:35:26.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.1 up 2024-09-10T22:35:26.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T22:35:26.318 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-09-10T22:35:26.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:26.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:26.395 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T22:35:26.410 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T22:35:26.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:26.410 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T22:35:26.463 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-10T22:35:26.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:26.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:26.546 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:26.546 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:26.554 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:26.554 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:26.554 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.94:3300,172.21.15.94:3301" 2024-09-10T22:35:26.554 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with new device syntax: 3@7ef137cd-a1ce-4819-b27d-b71268612f85.cephfs=/ 2024-09-10T22:35:26.555 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.66:3300/172.21.15.94:3300/172.21.15.94:3301" will pass to kernel 2024-09-10T22:35:26.555 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-10T22:35:26.555 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-09-10T22:35:26.556 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T22:35:26.556 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T22:35:26.556 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T22:35:26.557 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:35:26.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:26.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T22:35:26.621 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:26.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:26.717 INFO:teuthology.orchestra.run.smithi094.stdout:client4761 2024-09-10T22:35:26.717 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:26.717 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:26.733 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:26.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:26.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:26.829 INFO:teuthology.orchestra.run.smithi094.stdout:client4761 2024-09-10T22:35:26.829 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:26.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:26.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:26.887 INFO:teuthology.orchestra.run.smithi094.stdout:client4761 2024-09-10T22:35:26.887 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ef137cd-a1ce-4819-b27d-b71268612f85.client4761/status') 2024-09-10T22:35:26.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:26.944 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.4761 (3)172.21.15.94:0/4062368616 2024-09-10T22:35:26.944 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-09-10T22:35:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-09-10T22:35:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-09-10T22:35:26.945 INFO:teuthology.orchestra.run.smithi094.stderr:70 bytes copied, 1.9677e-05 s, 3.6 MB/s 2024-09-10T22:35:26.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-10T22:35:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:26.952 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T22:35:26.989 INFO:tasks.cephfs_test_runner:Starting test: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) The maximum number of openfiles omap objects keys are now equal to 2024-09-10T22:35:26.989 INFO:tasks.cephfs_test_runner:test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-09-10T22:35:26.989 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:27.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 -- 172.21.15.66:0/360439185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 msgr2=0x7f8b880b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/360439185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b880b3e70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b8809d980 tx=0x7f8b8402a560 comp rx=0 tx=0).stop 2024-09-10T22:35:27.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 -- 172.21.15.66:0/360439185 shutdown_connections 2024-09-10T22:35:27.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/360439185 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b880b43b0 0x7f8b88161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.111 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/360439185 >> v2:172.21.15.94:3300/0 conn(0x7f8b88162940 0x7f8b88161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/360439185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b880b3e70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 -- 172.21.15.66:0/360439185 >> 172.21.15.66:0/360439185 conn(0x7f8b8809d260 msgr2=0x7f8b880d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 -- 172.21.15.66:0/360439185 shutdown_connections 2024-09-10T22:35:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.107+0000 7f8b8f95d640 1 -- 172.21.15.66:0/360439185 wait complete. 2024-09-10T22:35:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.108+0000 7f8b8f95d640 1 Processor -- start 2024-09-10T22:35:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.108+0000 7f8b8f95d640 1 -- start start 2024-09-10T22:35:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8f95d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b880b43b0 0x7f8b881f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8f95d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b8814cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8f95d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b88162940 0x7f8b880b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8f95d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8b880d2110 con 0x7f8b88161f80 2024-09-10T22:35:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8f95d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8b880d1f90 con 0x7f8b88162940 2024-09-10T22:35:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8f95d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8b880d2290 con 0x7f8b880b43b0 2024-09-10T22:35:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8cef8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b8814cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8cef8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b8814cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46600/0 (socket says 172.21.15.66:46600) 2024-09-10T22:35:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8d6f9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b880b43b0 0x7f8b881f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8cef8640 1 -- 172.21.15.66:0/2008253000 learned_addr learned my addr 172.21.15.66:0/2008253000 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8defa640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b88162940 0x7f8b880b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8d6f9640 1 -- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b88162940 msgr2=0x7f8b880b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8d6f9640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b88162940 0x7f8b880b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8d6f9640 1 -- 172.21.15.66:0/2008253000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 msgr2=0x7f8b8814cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8d6f9640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b8814cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8d6f9640 1 -- 172.21.15.66:0/2008253000 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b881f60b0 con 0x7f8b880b43b0 2024-09-10T22:35:27.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.109+0000 7f8b8cef8640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b8814cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:27.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.110+0000 7f8b8d6f9640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b880b43b0 0x7f8b881f5b70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b84031000 tx=0x7f8b84003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:27.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.110+0000 7f8b7dffb640 1 -- 172.21.15.66:0/2008253000 <== mon.2 v2:172.21.15.94:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8403f070 con 0x7f8b880b43b0 2024-09-10T22:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.110+0000 7f8b7dffb640 1 -- 172.21.15.66:0/2008253000 <== mon.2 v2:172.21.15.94:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b840047a0 con 0x7f8b880b43b0 2024-09-10T22:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.110+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b880cd900 con 0x7f8b880b43b0 2024-09-10T22:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.110+0000 7f8b7dffb640 1 -- 172.21.15.66:0/2008253000 <== mon.2 v2:172.21.15.94:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84035030 con 0x7f8b880b43b0 2024-09-10T22:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.110+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b880cdec0 con 0x7f8b880b43b0 2024-09-10T22:35:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.111+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b880b3e70 con 0x7f8b880b43b0 2024-09-10T22:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.112+0000 7f8b7dffb640 1 -- 172.21.15.66:0/2008253000 <== mon.2 v2:172.21.15.94:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8b84004bb0 con 0x7f8b880b43b0 2024-09-10T22:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.112+0000 7f8b7dffb640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8b6c077f70 0x7f8b6c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.112+0000 7f8b7dffb640 1 -- 172.21.15.66:0/2008253000 <== mon.2 v2:172.21.15.94:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8b840bb240 con 0x7f8b880b43b0 2024-09-10T22:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.112+0000 7f8b8cef8640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8b6c077f70 0x7f8b6c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.113+0000 7f8b8cef8640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8b6c077f70 0x7f8b6c07a430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b881f2d00 tx=0x7f8b7800b410 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.117+0000 7f8b7dffb640 1 -- 172.21.15.66:0/2008253000 <== mon.2 v2:172.21.15.94:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8b84083760 con 0x7f8b880b43b0 2024-09-10T22:35:27.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.304+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b8808d6e0 con 0x7f8b880b43b0 2024-09-10T22:35:27.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.306+0000 7f8b7dffb640 1 -- 172.21.15.66:0/2008253000 <== mon.2 v2:172.21.15.94:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16641 (secure 0 0 0) 0x7f8b84088190 con 0x7f8b880b43b0 2024-09-10T22:35:27.308 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:27.308 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:04.052417+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:27.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8b6c077f70 msgr2=0x7f8b6c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8b6c077f70 0x7f8b6c07a430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b881f2d00 tx=0x7f8b7800b410 comp rx=0 tx=0).stop 2024-09-10T22:35:27.310 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b880b43b0 msgr2=0x7f8b881f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b880b43b0 0x7f8b881f5b70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b84031000 tx=0x7f8b84003910 comp rx=0 tx=0).stop 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 shutdown_connections 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8b6c077f70 0x7f8b6c07a430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8b88162940 0x7f8b880b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b88161f80 0x7f8b8814cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 --2- 172.21.15.66:0/2008253000 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8b880b43b0 0x7f8b881f5b70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 >> 172.21.15.66:0/2008253000 conn(0x7f8b8809d260 msgr2=0x7f8b880cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 shutdown_connections 2024-09-10T22:35:27.311 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.308+0000 7f8b8f95d640 1 -- 172.21.15.66:0/2008253000 wait complete. 2024-09-10T22:35:27.320 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:34:50.607631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:35:00.287286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:35:01.380372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:35:27.321 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:35:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 -- 172.21.15.66:0/544620362 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d1f90 msgr2=0x7fb0cc0d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 --2- 172.21.15.66:0/544620362 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d1f90 0x7fb0cc0d2420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4005fd0 tx=0x7fb0c402a780 comp rx=0 tx=0).stop 2024-09-10T22:35:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 -- 172.21.15.66:0/544620362 shutdown_connections 2024-09-10T22:35:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 --2- 172.21.15.66:0/544620362 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d1f90 0x7fb0cc0d2420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 --2- 172.21.15.66:0/544620362 >> v2:172.21.15.94:3300/0 conn(0x7fb0cc15e120 0x7fb0cc0d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 -- 172.21.15.66:0/544620362 >> 172.21.15.66:0/544620362 conn(0x7fb0cc0d3f10 msgr2=0x7fb0cc0d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:27.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 -- 172.21.15.66:0/544620362 shutdown_connections 2024-09-10T22:35:27.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.428+0000 7fb0d2382640 1 -- 172.21.15.66:0/544620362 wait complete. 2024-09-10T22:35:27.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.429+0000 7fb0d2382640 1 Processor -- start 2024-09-10T22:35:27.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.429+0000 7fb0d2382640 1 -- start start 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d2382640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d10d0 0x7fb0cc1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d2382640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0cc0d1f90 0x7fb0cc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d2382640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc15e120 0x7fb0cc0b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d2382640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc0b70f0 con 0x7fb0cc15e120 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d2382640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc0b6f70 con 0x7fb0cc0d1f90 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d2382640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb0cc0b7270 con 0x7fb0cc0d10d0 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc15e120 0x7fb0cc0b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc15e120 0x7fb0cc0b9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46602/0 (socket says 172.21.15.66:46602) 2024-09-10T22:35:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 -- 172.21.15.66:0/1415343298 learned_addr learned my addr 172.21.15.66:0/1415343298 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0cbfff640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d10d0 0x7fb0cc1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0cb7fe640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0cc0d1f90 0x7fb0cc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 -- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d10d0 msgr2=0x7fb0cc1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d10d0 0x7fb0cc1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 -- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0cc0d1f90 msgr2=0x7fb0cc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0cc0d1f90 0x7fb0cc15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.430+0000 7fb0d091f640 1 -- 172.21.15.66:0/1415343298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc1f4420 con 0x7fb0cc15e120 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.431+0000 7fb0d091f640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc15e120 0x7fb0cc0b9300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006010 tx=0x7fb0c4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.431+0000 7fb0c8ff9640 1 -- 172.21.15.66:0/1415343298 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4040070 con 0x7fb0cc15e120 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.431+0000 7fb0c8ff9640 1 -- 172.21.15.66:0/1415343298 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0c4004560 con 0x7fb0cc15e120 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.431+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc1f46e0 con 0x7fb0cc15e120 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.431+0000 7fb0c8ff9640 1 -- 172.21.15.66:0/1415343298 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4003e10 con 0x7fb0cc15e120 2024-09-10T22:35:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.431+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc1f4ca0 con 0x7fb0cc15e120 2024-09-10T22:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.432+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc0b7800 con 0x7fb0cc15e120 2024-09-10T22:35:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.433+0000 7fb0c8ff9640 1 -- 172.21.15.66:0/1415343298 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb0c403c420 con 0x7fb0cc15e120 2024-09-10T22:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.433+0000 7fb0c8ff9640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0b8077f70 0x7fb0b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.433+0000 7fb0c8ff9640 1 -- 172.21.15.66:0/1415343298 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb0c40bb040 con 0x7fb0cc15e120 2024-09-10T22:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.434+0000 7fb0cbfff640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0b8077f70 0x7fb0b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.434+0000 7fb0cbfff640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0b8077f70 0x7fb0b807a430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc158be0 tx=0x7fb0bc005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:27.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.437+0000 7fb0c8ff9640 1 -- 172.21.15.66:0/1415343298 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb0c40835c0 con 0x7fb0cc15e120 2024-09-10T22:35:27.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.620+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0cc076010 con 0x7fb0cc15e120 2024-09-10T22:35:27.623 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.621+0000 7fb0c8ff9640 1 -- 172.21.15.66:0/1415343298 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fb0c408c950 con 0x7fb0cc15e120 2024-09-10T22:35:27.623 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:35:27.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0b8077f70 msgr2=0x7fb0b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0b8077f70 0x7fb0b807a430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc158be0 tx=0x7fb0bc005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc15e120 msgr2=0x7fb0cc0b9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc15e120 0x7fb0cc0b9300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4006010 tx=0x7fb0c4003c20 comp rx=0 tx=0).stop 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 shutdown_connections 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb0b8077f70 0x7fb0b807a430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc15e120 0x7fb0cc0b9300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0cc0d1f90 0x7fb0cc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 --2- 172.21.15.66:0/1415343298 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb0cc0d10d0 0x7fb0cc1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 >> 172.21.15.66:0/1415343298 conn(0x7fb0cc0d3f10 msgr2=0x7fb0cc0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 shutdown_connections 2024-09-10T22:35:27.626 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.623+0000 7fb0d2382640 1 -- 172.21.15.66:0/1415343298 wait complete. 2024-09-10T22:35:27.636 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T22:35:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.742+0000 7ff3c5e71640 1 -- 172.21.15.66:0/3286829115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a8004d40 msgr2=0x7ff3a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.742+0000 7ff3c5e71640 1 --2- 172.21.15.66:0/3286829115 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3a8004d40 0x7ff3a80051c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005fd0 tx=0x7ff3a8005830 comp rx=0 tx=0).stop 2024-09-10T22:35:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.742+0000 7ff3bffff640 1 -- 172.21.15.66:0/3286829115 reap_dead start 2024-09-10T22:35:27.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.742+0000 7ff3c5e71640 1 -- 172.21.15.66:0/3286829115 shutdown_connections 2024-09-10T22:35:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.742+0000 7ff3c5e71640 1 -- 172.21.15.66:0/3286829115 >> 172.21.15.66:0/3286829115 conn(0x7ff3c00d1db0 msgr2=0x7ff3c00d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.742+0000 7ff3c5e71640 1 -- 172.21.15.66:0/3286829115 shutdown_connections 2024-09-10T22:35:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.742+0000 7ff3c5e71640 1 -- 172.21.15.66:0/3286829115 wait complete. 2024-09-10T22:35:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.743+0000 7ff3c5e71640 1 Processor -- start 2024-09-10T22:35:27.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.743+0000 7ff3c5e71640 1 -- start start 2024-09-10T22:35:27.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3c5e71640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c01533f0 0x7ff3c015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3c5e71640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00c78b0 0x7ff3c015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3c5e71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cb770 0x7ff3c01f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3c5e71640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00b2280 con 0x7ff3c00cb770 2024-09-10T22:35:27.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3c5e71640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00b2100 con 0x7ff3c01533f0 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3c5e71640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c00af920 con 0x7ff3c00c78b0 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cb770 0x7ff3c01f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3bffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cb770 0x7ff3c01f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46612/0 (socket says 172.21.15.66:46612) 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3bffff640 1 -- 172.21.15.66:0/274732689 learned_addr learned my addr 172.21.15.66:0/274732689 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3bf7fe640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c01533f0 0x7ff3c015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.744+0000 7ff3beffd640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00c78b0 0x7ff3c015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3bffff640 1 -- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00c78b0 msgr2=0x7ff3c015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3bffff640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00c78b0 0x7ff3c015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3bffff640 1 -- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c01533f0 msgr2=0x7ff3c015af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3bffff640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c01533f0 0x7ff3c015af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3bffff640 1 -- 172.21.15.66:0/274732689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c00b74d0 con 0x7ff3c00cb770 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3bffff640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cb770 0x7ff3c01f5fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff3b802b320 tx=0x7ff3b8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3a7fff640 1 -- 172.21.15.66:0/274732689 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8038040 con 0x7ff3c00cb770 2024-09-10T22:35:27.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3a7fff640 1 -- 172.21.15.66:0/274732689 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3b8003970 con 0x7ff3c00cb770 2024-09-10T22:35:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c00b77b0 con 0x7ff3c00cb770 2024-09-10T22:35:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3a7fff640 1 -- 172.21.15.66:0/274732689 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8003dc0 con 0x7ff3c00cb770 2024-09-10T22:35:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.745+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c00cb470 con 0x7ff3c00cb770 2024-09-10T22:35:27.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.746+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c00afee0 con 0x7ff3c00cb770 2024-09-10T22:35:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.746+0000 7ff3a7fff640 1 -- 172.21.15.66:0/274732689 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff3b803b070 con 0x7ff3c00cb770 2024-09-10T22:35:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.747+0000 7ff3a7fff640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3ac077fa0 0x7ff3ac07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.747+0000 7ff3a7fff640 1 -- 172.21.15.66:0/274732689 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b80b9460 con 0x7ff3c00cb770 2024-09-10T22:35:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.747+0000 7ff3bf7fe640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3ac077fa0 0x7ff3ac07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:27.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.747+0000 7ff3bf7fe640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3ac077fa0 0x7ff3ac07a460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff3c015bc40 tx=0x7ff3b000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:27.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.751+0000 7ff3a7fff640 1 -- 172.21.15.66:0/274732689 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff3b8081910 con 0x7ff3c00cb770 2024-09-10T22:35:27.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.935+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3c015a9e0 con 0x7ff3c00cb770 2024-09-10T22:35:27.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.935+0000 7ff3a7fff640 1 -- 172.21.15.66:0/274732689 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7ff3b8086340 con 0x7ff3c00cb770 2024-09-10T22:35:27.937 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2024-09-10T22:35:27.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3ac077fa0 msgr2=0x7ff3ac07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.939 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3ac077fa0 0x7ff3ac07a460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff3c015bc40 tx=0x7ff3b000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cb770 msgr2=0x7ff3c01f5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cb770 0x7ff3c01f5fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff3b802b320 tx=0x7ff3b8033670 comp rx=0 tx=0).stop 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 shutdown_connections 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3ac077fa0 0x7ff3ac07a460 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3c00cb770 0x7ff3c01f5fb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3c00c78b0 0x7ff3c015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 --2- 172.21.15.66:0/274732689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3c01533f0 0x7ff3c015af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 >> 172.21.15.66:0/274732689 conn(0x7ff3c00d1db0 msgr2=0x7ff3c015cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 shutdown_connections 2024-09-10T22:35:27.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:27.937+0000 7ff3c5e71640 1 -- 172.21.15.66:0/274732689 wait complete. 2024-09-10T22:35:27.950 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T22:35:28.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 -- 172.21.15.66:0/2091592116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 msgr2=0x7ff8980cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 --2- 172.21.15.66:0/2091592116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980cc530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff894009830 tx=0x7ff89402a510 comp rx=0 tx=0).stop 2024-09-10T22:35:28.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 -- 172.21.15.66:0/2091592116 shutdown_connections 2024-09-10T22:35:28.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 --2- 172.21.15.66:0/2091592116 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff89815acf0 0x7ff898163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.059 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 --2- 172.21.15.66:0/2091592116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980cc530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 -- 172.21.15.66:0/2091592116 >> 172.21.15.66:0/2091592116 conn(0x7ff8980b16d0 msgr2=0x7ff8980b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:28.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 -- 172.21.15.66:0/2091592116 shutdown_connections 2024-09-10T22:35:28.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.057+0000 7ff89fd32640 1 -- 172.21.15.66:0/2091592116 wait complete. 2024-09-10T22:35:28.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.058+0000 7ff89fd32640 1 Processor -- start 2024-09-10T22:35:28.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.058+0000 7ff89fd32640 1 -- start start 2024-09-10T22:35:28.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89fd32640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980cca70 0x7ff898150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89fd32640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89fd32640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff89815acf0 0x7ff8980c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89fd32640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff8980707f0 con 0x7ff898154370 2024-09-10T22:35:28.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89fd32640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff898070670 con 0x7ff8980cca70 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89fd32640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff898164060 con 0x7ff89815acf0 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46640/0 (socket says 172.21.15.66:46640) 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 -- 172.21.15.66:0/428553273 learned_addr learned my addr 172.21.15.66:0/428553273 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89e2cf640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff89815acf0 0x7ff8980c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89dace640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980cca70 0x7ff898150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 -- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff89815acf0 msgr2=0x7ff8980c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff89815acf0 0x7ff8980c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 -- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980cca70 msgr2=0x7ff898150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980cca70 0x7ff898150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 -- 172.21.15.66:0/428553273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89815c750 con 0x7ff898154370 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff89d2cd640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980c6cf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff88800c500 tx=0x7ff88800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.059+0000 7ff88e7fc640 1 -- 172.21.15.66:0/428553273 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888017070 con 0x7ff898154370 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.060+0000 7ff88e7fc640 1 -- 172.21.15.66:0/428553273 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8880035f0 con 0x7ff898154370 2024-09-10T22:35:28.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.060+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89815ca30 con 0x7ff898154370 2024-09-10T22:35:28.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.060+0000 7ff88e7fc640 1 -- 172.21.15.66:0/428553273 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888003a40 con 0x7ff898154370 2024-09-10T22:35:28.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.060+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89815cff0 con 0x7ff898154370 2024-09-10T22:35:28.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.061+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8980cc530 con 0x7ff898154370 2024-09-10T22:35:28.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.061+0000 7ff88e7fc640 1 -- 172.21.15.66:0/428553273 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff888003be0 con 0x7ff898154370 2024-09-10T22:35:28.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.062+0000 7ff88e7fc640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff87c077fa0 0x7ff87c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.062+0000 7ff88e7fc640 1 -- 172.21.15.66:0/428553273 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff8880993d0 con 0x7ff898154370 2024-09-10T22:35:28.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.062+0000 7ff89dace640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff87c077fa0 0x7ff87c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.063+0000 7ff89dace640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff87c077fa0 0x7ff87c07a460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff894034000 tx=0x7ff894004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:28.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.066+0000 7ff88e7fc640 1 -- 172.21.15.66:0/428553273 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff888061910 con 0x7ff898154370 2024-09-10T22:35:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.253+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff8980c9eb0 con 0x7ff898154370 2024-09-10T22:35:28.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.253+0000 7ff88e7fc640 1 -- 172.21.15.66:0/428553273 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff888066340 con 0x7ff898154370 2024-09-10T22:35:28.256 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2024-09-10T22:35:28.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff87c077fa0 msgr2=0x7ff87c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff87c077fa0 0x7ff87c07a460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff894034000 tx=0x7ff894004730 comp rx=0 tx=0).stop 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 msgr2=0x7ff8980c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980c6cf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff88800c500 tx=0x7ff88800c9c0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 shutdown_connections 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff87c077fa0 0x7ff87c07a460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff89815acf0 0x7ff8980c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff898154370 0x7ff8980c6cf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 --2- 172.21.15.66:0/428553273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8980cca70 0x7ff898150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 >> 172.21.15.66:0/428553273 conn(0x7ff8980b16d0 msgr2=0x7ff8980cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 shutdown_connections 2024-09-10T22:35:28.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.256+0000 7ff89fd32640 1 -- 172.21.15.66:0/428553273 wait complete. 2024-09-10T22:35:28.269 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-09-10T22:35:28.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.372+0000 7fe1f1a65640 1 -- 172.21.15.66:0/697852769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d40052e0 msgr2=0x7fe1d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.372+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/697852769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d40052e0 0x7fe1d400b9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4001d90 tx=0x7fe1d4005830 comp rx=0 tx=0).stop 2024-09-10T22:35:28.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.372+0000 7fe1f1a65640 1 -- 172.21.15.66:0/697852769 shutdown_connections 2024-09-10T22:35:28.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.372+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/697852769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1d40052e0 0x7fe1d400b9c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.372+0000 7fe1f1a65640 1 -- 172.21.15.66:0/697852769 >> 172.21.15.66:0/697852769 conn(0x7fe1ec0d0c90 msgr2=0x7fe1ec0d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:28.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.372+0000 7fe1f1a65640 1 -- 172.21.15.66:0/697852769 shutdown_connections 2024-09-10T22:35:28.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.373+0000 7fe1f1a65640 1 -- 172.21.15.66:0/697852769 wait complete. 2024-09-10T22:35:28.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.374+0000 7fe1f1a65640 1 Processor -- start 2024-09-10T22:35:28.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.374+0000 7fe1f1a65640 1 -- start start 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1f1a65640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1d40052e0 0x7fe1ec1efc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1f1a65640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ec0c6970 0x7fe1ec0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1f1a65640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1ec0c4980 0x7fe1ec0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1f1a65640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec071bf0 con 0x7fe1ec0c4980 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1f1a65640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec071a70 con 0x7fe1ec0c6970 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1f1a65640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ec071d70 con 0x7fe1d40052e0 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1ec0c4980 0x7fe1ec0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1ec0c4980 0x7fe1ec0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46654/0 (socket says 172.21.15.66:46654) 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 -- 172.21.15.66:0/3456022268 learned_addr learned my addr 172.21.15.66:0/3456022268 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1eb7fe640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1d40052e0 0x7fe1ec1efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1eaffd640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ec0c6970 0x7fe1ec0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 -- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1d40052e0 msgr2=0x7fe1ec1efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1d40052e0 0x7fe1ec1efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 -- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ec0c6970 msgr2=0x7fe1ec0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ec0c6970 0x7fe1ec0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 -- 172.21.15.66:0/3456022268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec0c74c0 con 0x7fe1ec0c4980 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1eaffd640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ec0c6970 0x7fe1ec0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1eb7fe640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1d40052e0 0x7fe1ec1efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1ebfff640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1ec0c4980 0x7fe1ec0c4e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4001d90 tx=0x7fe1e4006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1d3fff640 1 -- 172.21.15.66:0/3456022268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4003d80 con 0x7fe1ec0c4980 2024-09-10T22:35:28.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1d3fff640 1 -- 172.21.15.66:0/3456022268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e4005b80 con 0x7fe1ec0c4980 2024-09-10T22:35:28.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec0c77a0 con 0x7fe1ec0c4980 2024-09-10T22:35:28.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.375+0000 7fe1d3fff640 1 -- 172.21.15.66:0/3456022268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4005fd0 con 0x7fe1ec0c4980 2024-09-10T22:35:28.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.376+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1f01c0 con 0x7fe1ec0c4980 2024-09-10T22:35:28.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.376+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec06cea0 con 0x7fe1ec0c4980 2024-09-10T22:35:28.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.377+0000 7fe1d3fff640 1 -- 172.21.15.66:0/3456022268 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe1e40050d0 con 0x7fe1ec0c4980 2024-09-10T22:35:28.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.377+0000 7fe1d3fff640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1d8077fa0 0x7fe1d807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:28.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.378+0000 7fe1d3fff640 1 -- 172.21.15.66:0/3456022268 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1e40b9860 con 0x7fe1ec0c4980 2024-09-10T22:35:28.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.378+0000 7fe1eb7fe640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1d8077fa0 0x7fe1d807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:28.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.378+0000 7fe1eb7fe640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1d8077fa0 0x7fe1d807a460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0c8630 tx=0x7fe1dc006970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:28.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.382+0000 7fe1d3fff640 1 -- 172.21.15.66:0/3456022268 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe1e4043060 con 0x7fe1ec0c4980 2024-09-10T22:35:28.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:28.637+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) -- 0x7fe1ec15f570 con 0x7fe1ec0c4980 2024-09-10T22:35:29.126 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.124+0000 7fe1d3fff640 1 -- 172.21.15.66:0/3456022268 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v45) ==== 150+0+0 (secure 0 0 0) 0x7fe1e40b9070 con 0x7fe1ec0c4980 2024-09-10T22:35:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1d8077fa0 msgr2=0x7fe1d807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.129 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1d8077fa0 0x7fe1d807a460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0c8630 tx=0x7fe1dc006970 comp rx=0 tx=0).stop 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1ec0c4980 msgr2=0x7fe1ec0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1ec0c4980 0x7fe1ec0c4e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4001d90 tx=0x7fe1e4006ce0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 shutdown_connections 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1d8077fa0 0x7fe1d807a460 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1ec0c4980 0x7fe1ec0c4e30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ec0c6970 0x7fe1ec0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 --2- 172.21.15.66:0/3456022268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1d40052e0 0x7fe1ec1efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 >> 172.21.15.66:0/3456022268 conn(0x7fe1ec0d0c90 msgr2=0x7fe1ec15ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 shutdown_connections 2024-09-10T22:35:29.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.127+0000 7fe1f1a65640 1 -- 172.21.15.66:0/3456022268 wait complete. 2024-09-10T22:35:29.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.252+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/2367463692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d1ae0 msgr2=0x7fa6c00b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.252+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/2367463692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d1ae0 0x7fa6c00b4450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00a1e30 tx=0x7fa6b802aac0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.252+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/2367463692 shutdown_connections 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.252+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/2367463692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d1ae0 0x7fa6c00b4450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.252+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/2367463692 >> v2:172.21.15.94:3300/0 conn(0x7fa6c00d2be0 0x7fa6c00d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.252+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/2367463692 >> 172.21.15.66:0/2367463692 conn(0x7fa6c00d3f10 msgr2=0x7fa6c00d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.252+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/2367463692 shutdown_connections 2024-09-10T22:35:29.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.253+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/2367463692 wait complete. 2024-09-10T22:35:29.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 Processor -- start 2024-09-10T22:35:29.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 -- start start 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6c00d1ae0 0x7fa6c01f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d2be0 0x7fa6c00ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6c015e090 0x7fa6c00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00b4d20 con 0x7fa6c00d2be0 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c00b4ba0 con 0x7fa6c00d1ae0 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.254+0000 7fa6c7b6b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c00b4ea0 con 0x7fa6c015e090 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d2be0 0x7fa6c00ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d2be0 0x7fa6c00ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46682/0 (socket says 172.21.15.66:46682) 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 -- 172.21.15.66:0/3675287370 learned_addr learned my addr 172.21.15.66:0/3675287370 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5907640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6c00d1ae0 0x7fa6c01f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c6108640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6c015e090 0x7fa6c00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 -- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6c015e090 msgr2=0x7fa6c00ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6c015e090 0x7fa6c00ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 -- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6c00d1ae0 msgr2=0x7fa6c01f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6c00d1ae0 0x7fa6c01f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 -- 172.21.15.66:0/3675287370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c015cd30 con 0x7fa6c00d2be0 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c6108640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6c015e090 0x7fa6c00ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c5106640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d2be0 0x7fa6c00ccaa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000d950 tx=0x7fa6b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6b67fc640 1 -- 172.21.15.66:0/3675287370 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0017030 con 0x7fa6c00d2be0 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6b67fc640 1 -- 172.21.15.66:0/3675287370 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6b0009b20 con 0x7fa6c00d2be0 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c00cd6b0 con 0x7fa6c00d2be0 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6b67fc640 1 -- 172.21.15.66:0/3675287370 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0004020 con 0x7fa6c00d2be0 2024-09-10T22:35:29.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.255+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c014f4b0 con 0x7fa6c00d2be0 2024-09-10T22:35:29.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.256+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c00b4450 con 0x7fa6c00d2be0 2024-09-10T22:35:29.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.257+0000 7fa6b67fc640 1 -- 172.21.15.66:0/3675287370 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa6b00041c0 con 0x7fa6c00d2be0 2024-09-10T22:35:29.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.258+0000 7fa6b67fc640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa6a4077f70 0x7fa6a407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.258+0000 7fa6b67fc640 1 -- 172.21.15.66:0/3675287370 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6b0099e30 con 0x7fa6c00d2be0 2024-09-10T22:35:29.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.258+0000 7fa6c5907640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa6a4077f70 0x7fa6a407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.261 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.258+0000 7fa6c5907640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa6a4077f70 0x7fa6a407a430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00b38a0 tx=0x7fa6bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:29.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.262+0000 7fa6b67fc640 1 -- 172.21.15.66:0/3675287370 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa6c015cd30 con 0x7fa6c00d2be0 2024-09-10T22:35:29.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.447+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa6c00d03e0 con 0x7fa6c00d2be0 2024-09-10T22:35:29.449 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.447+0000 7fa6b67fc640 1 -- 172.21.15.66:0/3675287370 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7fa6b00629c0 con 0x7fa6c00d2be0 2024-09-10T22:35:29.449 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:35:29.449 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5584.179480 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4284.998881 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4476.318438 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4656.597806 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5031.347720 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4753.193614 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5217.773028 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4968.712004 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:35:29.450 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:35:29.451 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:35:29.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa6a4077f70 msgr2=0x7fa6a407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa6a4077f70 0x7fa6a407a430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00b38a0 tx=0x7fa6bc009040 comp rx=0 tx=0).stop 2024-09-10T22:35:29.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d2be0 msgr2=0x7fa6c00ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d2be0 0x7fa6c00ccaa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000d950 tx=0x7fa6b000de10 comp rx=0 tx=0).stop 2024-09-10T22:35:29.452 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 shutdown_connections 2024-09-10T22:35:29.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa6a4077f70 0x7fa6a407a430 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa6c015e090 0x7fa6c00ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa6c00d2be0 0x7fa6c00ccaa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 --2- 172.21.15.66:0/3675287370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa6c00d1ae0 0x7fa6c01f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.449+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 >> 172.21.15.66:0/3675287370 conn(0x7fa6c00d3f10 msgr2=0x7fa6c00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:29.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.450+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 shutdown_connections 2024-09-10T22:35:29.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.450+0000 7fa6c7b6b640 1 -- 172.21.15.66:0/3675287370 wait complete. 2024-09-10T22:35:29.462 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:35:29.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 -- 172.21.15.66:0/3043091249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c0b2d70 msgr2=0x7ff23c0b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 --2- 172.21.15.66:0/3043091249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c0b2d70 0x7ff23c0b1bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff224009830 tx=0x7ff22402a510 comp rx=0 tx=0).stop 2024-09-10T22:35:29.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 -- 172.21.15.66:0/3043091249 shutdown_connections 2024-09-10T22:35:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 --2- 172.21.15.66:0/3043091249 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff23c0af490 0x7ff23c0af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 --2- 172.21.15.66:0/3043091249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c0b2d70 0x7ff23c0b1bc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 -- 172.21.15.66:0/3043091249 >> 172.21.15.66:0/3043091249 conn(0x7ff23c0d1db0 msgr2=0x7ff23c0d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 -- 172.21.15.66:0/3043091249 shutdown_connections 2024-09-10T22:35:29.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.568+0000 7ff240e47640 1 -- 172.21.15.66:0/3043091249 wait complete. 2024-09-10T22:35:29.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.569+0000 7ff240e47640 1 Processor -- start 2024-09-10T22:35:29.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.569+0000 7ff240e47640 1 -- start start 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff240e47640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff23c0af490 0x7ff23c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff240e47640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff23c0b2d70 0x7ff23c15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff240e47640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c15f5b0 0x7ff23c15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff240e47640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0b33d0 con 0x7ff23c0b2d70 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff240e47640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0b2310 con 0x7ff23c15f5b0 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff240e47640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c0afe60 con 0x7ff23c0af490 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff233fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff23c0b2d70 0x7ff23c15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff233fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff23c0b2d70 0x7ff23c15e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46694/0 (socket says 172.21.15.66:46694) 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff233fff640 1 -- 172.21.15.66:0/1278193408 learned_addr learned my addr 172.21.15.66:0/1278193408 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff23ad9d640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c15f5b0 0x7ff23c15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.570+0000 7ff23a59c640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff23c0af490 0x7ff23c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff233fff640 1 -- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff23c0af490 msgr2=0x7ff23c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff233fff640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff23c0af490 0x7ff23c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff233fff640 1 -- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c15f5b0 msgr2=0x7ff23c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff233fff640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c15f5b0 0x7ff23c15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff233fff640 1 -- 172.21.15.66:0/1278193408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c1f2db0 con 0x7ff23c0b2d70 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff233fff640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff23c0b2d70 0x7ff23c15e420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff224009950 tx=0x7ff224032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff232ffd640 1 -- 172.21.15.66:0/1278193408 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224040070 con 0x7ff23c0b2d70 2024-09-10T22:35:29.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c1f3090 con 0x7ff23c0b2d70 2024-09-10T22:35:29.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff232ffd640 1 -- 172.21.15.66:0/1278193408 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff224005310 con 0x7ff23c0b2d70 2024-09-10T22:35:29.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff232ffd640 1 -- 172.21.15.66:0/1278193408 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224005c90 con 0x7ff23c0b2d70 2024-09-10T22:35:29.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.571+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1f3650 con 0x7ff23c0b2d70 2024-09-10T22:35:29.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.572+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c0af920 con 0x7ff23c0b2d70 2024-09-10T22:35:29.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.573+0000 7ff232ffd640 1 -- 172.21.15.66:0/1278193408 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff2240049b0 con 0x7ff23c0b2d70 2024-09-10T22:35:29.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.573+0000 7ff232ffd640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff21c077f70 0x7ff21c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:29.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.573+0000 7ff232ffd640 1 -- 172.21.15.66:0/1278193408 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2240bb180 con 0x7ff23c0b2d70 2024-09-10T22:35:29.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.573+0000 7ff23a59c640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff21c077f70 0x7ff21c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:29.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.574+0000 7ff23a59c640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff21c077f70 0x7ff21c07a430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff22c007fd0 tx=0x7ff22c001390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:29.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.577+0000 7ff232ffd640 1 -- 172.21.15.66:0/1278193408 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff224083c90 con 0x7ff23c0b2d70 2024-09-10T22:35:29.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.762+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff23c0d05c0 con 0x7ff23c0b2d70 2024-09-10T22:35:29.767 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.765+0000 7ff232ffd640 1 -- 172.21.15.66:0/1278193408 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7ff224083590 con 0x7ff23c0b2d70 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.767+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff21c077f70 msgr2=0x7ff21c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff21c077f70 0x7ff21c07a430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff22c007fd0 tx=0x7ff22c001390 comp rx=0 tx=0).stop 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff23c0b2d70 msgr2=0x7ff23c15e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff23c0b2d70 0x7ff23c15e420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff224009950 tx=0x7ff224032c10 comp rx=0 tx=0).stop 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 shutdown_connections 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff21c077f70 0x7ff21c07a430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff23c15f5b0 0x7ff23c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff23c0b2d70 0x7ff23c15e420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 --2- 172.21.15.66:0/1278193408 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff23c0af490 0x7ff23c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:29.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 >> 172.21.15.66:0/1278193408 conn(0x7ff23c0d1db0 msgr2=0x7ff23c0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:29.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 shutdown_connections 2024-09-10T22:35:29.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:29.768+0000 7ff240e47640 1 -- 172.21.15.66:0/1278193408 wait complete. 2024-09-10T22:35:29.780 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:35:29.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:35:29.820 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:35:29.871 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:35:29.871 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:29.871 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:29.887 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:29.887 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:29.943 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:29.944 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:30.000 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:35:30.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 -- 172.21.15.66:0/1011974827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 msgr2=0x7f1854073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 --2- 172.21.15.66:0/1011974827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854073460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1848009860 tx=0x7f184802a540 comp rx=0 tx=0).stop 2024-09-10T22:35:30.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 -- 172.21.15.66:0/1011974827 shutdown_connections 2024-09-10T22:35:30.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 --2- 172.21.15.66:0/1011974827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854073460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 --2- 172.21.15.66:0/1011974827 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f185406f890 0x7f18540cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 -- 172.21.15.66:0/1011974827 >> 172.21.15.66:0/1011974827 conn(0x7f18540b1660 msgr2=0x7f18540b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:30.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 -- 172.21.15.66:0/1011974827 shutdown_connections 2024-09-10T22:35:30.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.144+0000 7f185bcbd640 1 -- 172.21.15.66:0/1011974827 wait complete. 2024-09-10T22:35:30.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.145+0000 7f185bcbd640 1 Processor -- start 2024-09-10T22:35:30.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185bcbd640 1 -- start start 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185bcbd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185406f890 0x7f18540b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185bcbd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f185406fc70 0x7f18540b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185bcbd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185bcbd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f185415a770 con 0x7f185415e6e0 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185bcbd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1854073eb0 con 0x7f185406f890 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185bcbd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f185415a8f0 con 0x7f185406fc70 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185a25a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185a25a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46704/0 (socket says 172.21.15.66:46704) 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.146+0000 7f185a25a640 1 -- 172.21.15.66:0/972794395 learned_addr learned my addr 172.21.15.66:0/972794395 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f1859258640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f185406fc70 0x7f18540b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f1859a59640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185406f890 0x7f18540b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.150 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185a25a640 1 -- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f185406fc70 msgr2=0x7f18540b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185a25a640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f185406fc70 0x7f18540b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185a25a640 1 -- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185406f890 msgr2=0x7f18540b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185a25a640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185406f890 0x7f18540b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185a25a640 1 -- 172.21.15.66:0/972794395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18541f83f0 con 0x7f185415e6e0 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f1859258640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f185406fc70 0x7f18540b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f1859a59640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185406f890 0x7f18540b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185a25a640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854155770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f184400c680 tx=0x7f184400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f18527fc640 1 -- 172.21.15.66:0/972794395 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844017070 con 0x7f185415e6e0 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f18527fc640 1 -- 172.21.15.66:0/972794395 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1844003c60 con 0x7f185415e6e0 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541f86d0 con 0x7f185415e6e0 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f18527fc640 1 -- 172.21.15.66:0/972794395 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18440052d0 con 0x7f185415e6e0 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.147+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18541f8c90 con 0x7f185415e6e0 2024-09-10T22:35:30.151 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.148+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1854156b90 con 0x7f185415e6e0 2024-09-10T22:35:30.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.149+0000 7f18527fc640 1 -- 172.21.15.66:0/972794395 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1844005470 con 0x7f185415e6e0 2024-09-10T22:35:30.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.149+0000 7f18527fc640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1840077f70 0x7f184007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.149+0000 7f18527fc640 1 -- 172.21.15.66:0/972794395 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f18440995e0 con 0x7f185415e6e0 2024-09-10T22:35:30.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.150+0000 7f1859a59640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1840077f70 0x7f184007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.152 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.150+0000 7f1859a59640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1840077f70 0x7f184007a430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f184c009b40 tx=0x7f184c005e50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:30.155 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.154+0000 7f18527fc640 1 -- 172.21.15.66:0/972794395 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1844062170 con 0x7f185415e6e0 2024-09-10T22:35:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.349+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f18540cd870 con 0x7f185415e6e0 2024-09-10T22:35:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.349+0000 7f18527fc640 1 -- 172.21.15.66:0/972794395 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f1844061a70 con 0x7f185415e6e0 2024-09-10T22:35:30.351 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:35:30.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1840077f70 msgr2=0x7f184007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1840077f70 0x7f184007a430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f184c009b40 tx=0x7f184c005e50 comp rx=0 tx=0).stop 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 msgr2=0x7f1854155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854155770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f184400c680 tx=0x7f184400cb40 comp rx=0 tx=0).stop 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 shutdown_connections 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f1840077f70 0x7f184007a430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f185415e6e0 0x7f1854155770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f185406fc70 0x7f18540b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 --2- 172.21.15.66:0/972794395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185406f890 0x7f18540b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 >> 172.21.15.66:0/972794395 conn(0x7f18540b1660 msgr2=0x7f18540cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 shutdown_connections 2024-09-10T22:35:30.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.351+0000 7f185bcbd640 1 -- 172.21.15.66:0/972794395 wait complete. 2024-09-10T22:35:30.364 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T22:35:30.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:30.364 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T22:35:30.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:35:30.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:30.392 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:35:30.458 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:30.458 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:30.477 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:30.477 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:30.537 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:30.537 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:30.595 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:35:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 -- 172.21.15.66:0/960529295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 msgr2=0x7fb3740d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 --2- 172.21.15.66:0/960529295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 0x7fb3740d4a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb36c005fd0 tx=0x7fb36c02a980 comp rx=0 tx=0).stop 2024-09-10T22:35:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 -- 172.21.15.66:0/960529295 shutdown_connections 2024-09-10T22:35:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 --2- 172.21.15.66:0/960529295 >> v2:172.21.15.94:3300/0 conn(0x7fb3740d1f90 0x7fb3740d2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 --2- 172.21.15.66:0/960529295 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 0x7fb3740d4a30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 -- 172.21.15.66:0/960529295 >> 172.21.15.66:0/960529295 conn(0x7fb3740d3f10 msgr2=0x7fb3740d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:30.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 -- 172.21.15.66:0/960529295 shutdown_connections 2024-09-10T22:35:30.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.700+0000 7fb378f24640 1 -- 172.21.15.66:0/960529295 wait complete. 2024-09-10T22:35:30.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.701+0000 7fb378f24640 1 Processor -- start 2024-09-10T22:35:30.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.701+0000 7fb378f24640 1 -- start start 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb378f24640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3740d10d0 0x7fb3740cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb378f24640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3740d1f90 0x7fb3741553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb378f24640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 0x7fb37409f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb378f24640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb3740b70f0 con 0x7fb3740d1f90 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb378f24640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3740b6f70 con 0x7fb3740d10d0 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb378f24640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb3740b7270 con 0x7fb37415e120 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3740d1f90 0x7fb3741553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3740d1f90 0x7fb3741553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46710/0 (socket says 172.21.15.66:46710) 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 -- 172.21.15.66:0/1343885004 learned_addr learned my addr 172.21.15.66:0/1343885004 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:30.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb372d9d640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 0x7fb37409f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb37259c640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3740d10d0 0x7fb3740cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 -- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 msgr2=0x7fb37409f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 0x7fb37409f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 -- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3740d10d0 msgr2=0x7fb3740cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3740d10d0 0x7fb3740cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.702+0000 7fb371d9b640 1 -- 172.21.15.66:0/1343885004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb374155900 con 0x7fb3740d1f90 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.703+0000 7fb371d9b640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3740d1f90 0x7fb3741553c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c3d0 tx=0x7fb35c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.703+0000 7fb36affd640 1 -- 172.21.15.66:0/1343885004 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c017070 con 0x7fb3740d1f90 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.703+0000 7fb36affd640 1 -- 172.21.15.66:0/1343885004 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb35c0053c0 con 0x7fb3740d1f90 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.703+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3740cb5a0 con 0x7fb3740d1f90 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.703+0000 7fb36affd640 1 -- 172.21.15.66:0/1343885004 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0038a0 con 0x7fb3740d1f90 2024-09-10T22:35:30.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.703+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3740cbb60 con 0x7fb3740d1f90 2024-09-10T22:35:30.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.704+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3740d0b90 con 0x7fb3740d1f90 2024-09-10T22:35:30.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.704+0000 7fb36affd640 1 -- 172.21.15.66:0/1343885004 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb35c003a40 con 0x7fb3740d1f90 2024-09-10T22:35:30.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.705+0000 7fb36affd640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb354077f70 0x7fb35407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:30.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.705+0000 7fb36affd640 1 -- 172.21.15.66:0/1343885004 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb35c0992f0 con 0x7fb3740d1f90 2024-09-10T22:35:30.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.705+0000 7fb37259c640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb354077f70 0x7fb35407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:30.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.706+0000 7fb37259c640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb354077f70 0x7fb35407a430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb36c035000 tx=0x7fb36c004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:30.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.709+0000 7fb36affd640 1 -- 172.21.15.66:0/1343885004 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb3740d0b90 con 0x7fb3740d1f90 2024-09-10T22:35:30.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.916+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb3740b7800 con 0x7fb3740d1f90 2024-09-10T22:35:30.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.916+0000 7fb36affd640 1 -- 172.21.15.66:0/1343885004 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fb374155900 con 0x7fb3740d1f90 2024-09-10T22:35:30.918 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb354077f70 msgr2=0x7fb35407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb354077f70 0x7fb35407a430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb36c035000 tx=0x7fb36c004ab0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3740d1f90 msgr2=0x7fb3741553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3740d1f90 0x7fb3741553c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c3d0 tx=0x7fb35c00c890 comp rx=0 tx=0).stop 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 shutdown_connections 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb354077f70 0x7fb35407a430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb37415e120 0x7fb37409f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb3740d1f90 0x7fb3741553c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 --2- 172.21.15.66:0/1343885004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3740d10d0 0x7fb3740cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:30.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 >> 172.21.15.66:0/1343885004 conn(0x7fb3740d3f10 msgr2=0x7fb3740b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:30.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.918+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 shutdown_connections 2024-09-10T22:35:30.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:30.919+0000 7fb378f24640 1 -- 172.21.15.66:0/1343885004 wait complete. 2024-09-10T22:35:30.931 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T22:35:30.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:30.932 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T22:35:30.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:35:30.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:30.964 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T22:35:31.031 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:31.031 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:31.049 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:31.049 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:31.105 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:31.105 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:31.163 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 -- 172.21.15.66:0/2273760110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80cfac0 msgr2=0x7ff5e8161ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2273760110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80cfac0 0x7ff5e8161ef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806d260 tx=0x7ff5e002aac0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 -- 172.21.15.66:0/2273760110 shutdown_connections 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2273760110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80cfac0 0x7ff5e8161ef0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2273760110 >> v2:172.21.15.94:3300/0 conn(0x7ff5e80d4ad0 0x7ff5e80cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 -- 172.21.15.66:0/2273760110 >> 172.21.15.66:0/2273760110 conn(0x7ff5e80d3fb0 msgr2=0x7ff5e80d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 -- 172.21.15.66:0/2273760110 shutdown_connections 2024-09-10T22:35:31.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.318+0000 7ff5ed589640 1 -- 172.21.15.66:0/2273760110 wait complete. 2024-09-10T22:35:31.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.320+0000 7ff5ed589640 1 Processor -- start 2024-09-10T22:35:31.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.320+0000 7ff5ed589640 1 -- start start 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5ed589640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5e80cfac0 0x7ff5e8158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5ed589640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80d4ad0 0x7ff5e80ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5ed589640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5e8161b10 0x7ff5e8159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5ed589640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e8162430 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5ed589640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e80ceb00 con 0x7ff5e80cfac0 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5ed589640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e81625b0 con 0x7ff5e8161b10 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80d4ad0 0x7ff5e80ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80d4ad0 0x7ff5e80ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46734/0 (socket says 172.21.15.66:46734) 2024-09-10T22:35:31.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 -- 172.21.15.66:0/2623209127 learned_addr learned my addr 172.21.15.66:0/2623209127 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e6ffd640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5e80cfac0 0x7ff5e8158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e77fe640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5e8161b10 0x7ff5e8159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 -- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5e8161b10 msgr2=0x7ff5e8159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5e8161b10 0x7ff5e8159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 -- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5e80cfac0 msgr2=0x7ff5e8158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5e80cfac0 0x7ff5e8158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 -- 172.21.15.66:0/2623209127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e815bf20 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5e67fc640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80d4ad0 0x7ff5e80ccce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800d7a0 tx=0x7ff5d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.321+0000 7ff5df7fe640 1 -- 172.21.15.66:0/2623209127 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8017070 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.322+0000 7ff5df7fe640 1 -- 172.21.15.66:0/2623209127 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5d8009d20 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.322+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e815c200 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.322+0000 7ff5df7fe640 1 -- 172.21.15.66:0/2623209127 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8005280 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.322+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e80b7b90 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.323+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e80cf4f0 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.323+0000 7ff5df7fe640 1 -- 172.21.15.66:0/2623209127 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff5d8005420 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.324+0000 7ff5df7fe640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.324+0000 7ff5df7fe640 1 -- 172.21.15.66:0/2623209127 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff5d8099e10 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.324+0000 7ff5e6ffd640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.324+0000 7ff5e6ffd640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806ad80 tx=0x7ff5d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:31.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.328+0000 7ff5df7fe640 1 -- 172.21.15.66:0/2623209127 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff5e80cf4f0 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.528+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff5e8161ef0 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.528+0000 7ff5df7fe640 1 -- 172.21.15.66:0/2623209127 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7ff5d80622c0 con 0x7ff5e80d4ad0 2024-09-10T22:35:31.532 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:35:31.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.530+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 msgr2=0x7ff5c807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.530+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806ad80 tx=0x7ff5d0009040 comp rx=0 tx=0).stop 2024-09-10T22:35:31.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.530+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80d4ad0 msgr2=0x7ff5e80ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.530+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80d4ad0 0x7ff5e80ccce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800d7a0 tx=0x7ff5d800dc60 comp rx=0 tx=0).stop 2024-09-10T22:35:31.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 shutdown_connections 2024-09-10T22:35:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5e8161b10 0x7ff5e8159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5e80d4ad0 0x7ff5e80ccce0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 --2- 172.21.15.66:0/2623209127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5e80cfac0 0x7ff5e8158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 >> 172.21.15.66:0/2623209127 conn(0x7ff5e80d3fb0 msgr2=0x7ff5e80b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 shutdown_connections 2024-09-10T22:35:31.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.531+0000 7ff5ed589640 1 -- 172.21.15.66:0/2623209127 wait complete. 2024-09-10T22:35:31.544 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T22:35:31.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:31.544 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T22:35:31.572 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:35:31.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:31.572 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T22:35:31.587 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:35:31.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:35:31.628 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:31.629 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:31.677 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:31.677 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:31.737 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:35:31.737 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:31.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2610557235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c0b1630 msgr2=0x7f9b3c160450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2610557235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c0b1630 0x7f9b3c160450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009830 tx=0x7f9b2c02a510 comp rx=0 tx=0).stop 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2610557235 shutdown_connections 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2610557235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c0b1630 0x7f9b3c160450 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2610557235 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b3c068af0 0x7f9b3c0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2610557235 >> 172.21.15.66:0/2610557235 conn(0x7f9b3c0b1f00 msgr2=0x7f9b3c0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2610557235 shutdown_connections 2024-09-10T22:35:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.899+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2610557235 wait complete. 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 Processor -- start 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 -- start start 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b3c068af0 0x7f9b3c0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b3c0b1630 0x7f9b3c0c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c15be20 0x7f9b3c0c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c0cc6b0 con 0x7f9b3c15be20 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c0cc530 con 0x7f9b3c0b1630 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.901+0000 7f9b43ab8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3c0cc830 con 0x7f9b3c068af0 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c15be20 0x7f9b3c0c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c15be20 0x7f9b3c0c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46752/0 (socket says 172.21.15.66:46752) 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 -- 172.21.15.66:0/2897152009 learned_addr learned my addr 172.21.15.66:0/2897152009 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b41854640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b3c068af0 0x7f9b3c0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b41053640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b3c0b1630 0x7f9b3c0c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 -- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b3c068af0 msgr2=0x7f9b3c0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b3c068af0 0x7f9b3c0cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 -- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b3c0b1630 msgr2=0x7f9b3c0c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b3c0b1630 0x7f9b3c0c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 -- 172.21.15.66:0/2897152009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c0cffa0 con 0x7f9b3c15be20 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b41053640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b3c0b1630 0x7f9b3c0c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b41854640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b3c068af0 0x7f9b3c0cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b42055640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c15be20 0x7f9b3c0c9a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400eaa0 tx=0x7f9b3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b327fc640 1 -- 172.21.15.66:0/2897152009 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400cdb0 con 0x7f9b3c15be20 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b327fc640 1 -- 172.21.15.66:0/2897152009 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b34004c10 con 0x7f9b3c15be20 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3c0cf720 con 0x7f9b3c15be20 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b327fc640 1 -- 172.21.15.66:0/2897152009 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34005020 con 0x7f9b3c15be20 2024-09-10T22:35:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.902+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3c0cfcc0 con 0x7f9b3c15be20 2024-09-10T22:35:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.903+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3c0cbde0 con 0x7f9b3c15be20 2024-09-10T22:35:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.904+0000 7f9b327fc640 1 -- 172.21.15.66:0/2897152009 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9b3400f040 con 0x7f9b3c15be20 2024-09-10T22:35:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.904+0000 7f9b327fc640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b20077f70 0x7f9b2007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.905+0000 7f9b327fc640 1 -- 172.21.15.66:0/2897152009 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9b3409a4d0 con 0x7f9b3c15be20 2024-09-10T22:35:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.905+0000 7f9b41854640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b20077f70 0x7f9b2007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:31.908 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.905+0000 7f9b41854640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b20077f70 0x7f9b2007a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b380016a0 tx=0x7f9b38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:31.910 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:31.909+0000 7f9b327fc640 1 -- 172.21.15.66:0/2897152009 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9b34016030 con 0x7f9b3c15be20 2024-09-10T22:35:32.106 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.104+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9b3c0709b0 con 0x7f9b3c15be20 2024-09-10T22:35:32.107 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.105+0000 7f9b327fc640 1 -- 172.21.15.66:0/2897152009 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f9b34098e20 con 0x7f9b3c15be20 2024-09-10T22:35:32.107 INFO:teuthology.orchestra.run.smithi066.stderr:listed 0 entries 2024-09-10T22:35:32.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b20077f70 msgr2=0x7f9b2007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b20077f70 0x7f9b2007a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b380016a0 tx=0x7f9b38009040 comp rx=0 tx=0).stop 2024-09-10T22:35:32.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c15be20 msgr2=0x7f9b3c0c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c15be20 0x7f9b3c0c9a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400eaa0 tx=0x7f9b3400ef60 comp rx=0 tx=0).stop 2024-09-10T22:35:32.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 shutdown_connections 2024-09-10T22:35:32.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9b20077f70 0x7f9b2007a430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b3c15be20 0x7f9b3c0c9a30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b3c0b1630 0x7f9b3c0c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 --2- 172.21.15.66:0/2897152009 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9b3c068af0 0x7f9b3c0cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 >> 172.21.15.66:0/2897152009 conn(0x7f9b3c0b1f00 msgr2=0x7f9b3c0749f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:32.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 shutdown_connections 2024-09-10T22:35:32.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.107+0000 7f9b43ab8640 1 -- 172.21.15.66:0/2897152009 wait complete. 2024-09-10T22:35:32.120 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T22:35:32.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:32.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T22:35:32.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:35:32.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:32.152 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T22:35:32.210 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:35:32.210 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:32.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 -- 172.21.15.66:0/3140224545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 msgr2=0x7ff80c0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 --2- 172.21.15.66:0/3140224545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 0x7ff80c0ce860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff800009900 tx=0x7ff80002a610 comp rx=0 tx=0).stop 2024-09-10T22:35:32.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 -- 172.21.15.66:0/3140224545 shutdown_connections 2024-09-10T22:35:32.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 --2- 172.21.15.66:0/3140224545 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 0x7ff80c0ce860 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 --2- 172.21.15.66:0/3140224545 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c154e90 0x7ff80c06ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 -- 172.21.15.66:0/3140224545 >> 172.21.15.66:0/3140224545 conn(0x7ff80c0b16d0 msgr2=0x7ff80c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:32.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 -- 172.21.15.66:0/3140224545 shutdown_connections 2024-09-10T22:35:32.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.313+0000 7ff8122e6640 1 -- 172.21.15.66:0/3140224545 wait complete. 2024-09-10T22:35:32.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.315+0000 7ff8122e6640 1 Processor -- start 2024-09-10T22:35:32.319 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.315+0000 7ff8122e6640 1 -- start start 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.315+0000 7ff8122e6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c06f1f0 0x7ff80c14dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff8122e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c154e90 0x7ff80c0cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff8122e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 0x7ff80c1e8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff8122e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c0cf130 con 0x7ff80c154e90 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff8122e6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c0cefb0 con 0x7ff80c155270 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff8122e6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff80c15ed50 con 0x7ff80c06f1f0 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c154e90 0x7ff80c0cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c154e90 0x7ff80c0cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46766/0 (socket says 172.21.15.66:46766) 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 -- 172.21.15.66:0/1670351602 learned_addr learned my addr 172.21.15.66:0/1670351602 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff810883640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 0x7ff80c1e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80bfff640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c06f1f0 0x7ff80c14dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 -- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c06f1f0 msgr2=0x7ff80c14dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c06f1f0 0x7ff80c14dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 -- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 msgr2=0x7ff80c1e8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 0x7ff80c1e8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 -- 172.21.15.66:0/1670351602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c0cae80 con 0x7ff80c154e90 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff810883640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 0x7ff80c1e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff80b7fe640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c154e90 0x7ff80c0cc360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff800034040 tx=0x7ff800003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff808ff9640 1 -- 172.21.15.66:0/1670351602 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80003e030 con 0x7ff80c154e90 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff808ff9640 1 -- 172.21.15.66:0/1670351602 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff800003c20 con 0x7ff80c154e90 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c0cb120 con 0x7ff80c154e90 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.316+0000 7ff808ff9640 1 -- 172.21.15.66:0/1670351602 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff800035040 con 0x7ff80c154e90 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.317+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c0cb6e0 con 0x7ff80c154e90 2024-09-10T22:35:32.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.317+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80c0c6a10 con 0x7ff80c154e90 2024-09-10T22:35:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.318+0000 7ff808ff9640 1 -- 172.21.15.66:0/1670351602 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff800031400 con 0x7ff80c154e90 2024-09-10T22:35:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.319+0000 7ff808ff9640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff7f8077fa0 0x7ff7f807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.319+0000 7ff808ff9640 1 -- 172.21.15.66:0/1670351602 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff8000ba2b0 con 0x7ff80c154e90 2024-09-10T22:35:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.319+0000 7ff80bfff640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff7f8077fa0 0x7ff7f807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.319+0000 7ff80bfff640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff7f8077fa0 0x7ff7f807a460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009bd0 tx=0x7ff7fc005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:32.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.323+0000 7ff808ff9640 1 -- 172.21.15.66:0/1670351602 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff80c0c6a10 con 0x7ff80c154e90 2024-09-10T22:35:32.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.542+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff80c0c9e90 con 0x7ff80c154e90 2024-09-10T22:35:32.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.543+0000 7ff808ff9640 1 -- 172.21.15.66:0/1670351602 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7ff8000827f0 con 0x7ff80c154e90 2024-09-10T22:35:32.545 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:32.545 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:32.545 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff7f8077fa0 msgr2=0x7ff7f807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff7f8077fa0 0x7ff7f807a460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009bd0 tx=0x7ff7fc005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:32.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c154e90 msgr2=0x7ff80c0cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c154e90 0x7ff80c0cc360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff800034040 tx=0x7ff800003820 comp rx=0 tx=0).stop 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 shutdown_connections 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff7f8077fa0 0x7ff7f807a460 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff80c155270 0x7ff80c1e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff80c154e90 0x7ff80c0cc360 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 --2- 172.21.15.66:0/1670351602 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff80c06f1f0 0x7ff80c14dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 >> 172.21.15.66:0/1670351602 conn(0x7ff80c0b16d0 msgr2=0x7ff80c072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 shutdown_connections 2024-09-10T22:35:32.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.545+0000 7ff8122e6640 1 -- 172.21.15.66:0/1670351602 wait complete. 2024-09-10T22:35:32.558 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.662+0000 7ff58d239640 1 -- 172.21.15.66:0/2764798542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff574004800 msgr2=0x7ff574004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.662+0000 7ff58d239640 1 --2- 172.21.15.66:0/2764798542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff574004800 0x7ff574004c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c2e0 tx=0x7ff574005830 comp rx=0 tx=0).stop 2024-09-10T22:35:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.662+0000 7ff58d239640 1 -- 172.21.15.66:0/2764798542 shutdown_connections 2024-09-10T22:35:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.662+0000 7ff58d239640 1 --2- 172.21.15.66:0/2764798542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff574004800 0x7ff574004c80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.662+0000 7ff58d239640 1 -- 172.21.15.66:0/2764798542 >> 172.21.15.66:0/2764798542 conn(0x7ff5880d3f10 msgr2=0x7ff5880d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.662+0000 7ff58d239640 1 -- 172.21.15.66:0/2764798542 shutdown_connections 2024-09-10T22:35:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.662+0000 7ff58d239640 1 -- 172.21.15.66:0/2764798542 wait complete. 2024-09-10T22:35:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.663+0000 7ff58d239640 1 Processor -- start 2024-09-10T22:35:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.663+0000 7ff58d239640 1 -- start start 2024-09-10T22:35:32.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58d239640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff574004800 0x7ff58815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58d239640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5880b7750 0x7ff58815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58d239640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588156f90 0x7ff588157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58d239640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5880d1e70 con 0x7ff5880b7750 2024-09-10T22:35:32.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58d239640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5880d1cf0 con 0x7ff588156f90 2024-09-10T22:35:32.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58d239640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff5880d24d0 con 0x7ff574004800 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5880b7750 0x7ff58815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5880b7750 0x7ff58815acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46792/0 (socket says 172.21.15.66:46792) 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 -- 172.21.15.66:0/1714096270 learned_addr learned my addr 172.21.15.66:0/1714096270 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff586d9d640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff574004800 0x7ff58815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58759e640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588156f90 0x7ff588157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 -- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff574004800 msgr2=0x7ff58815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff574004800 0x7ff58815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 -- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588156f90 msgr2=0x7ff588157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588156f90 0x7ff588157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58659c640 1 -- 172.21.15.66:0/1714096270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5880b7d10 con 0x7ff5880b7750 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.664+0000 7ff58759e640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588156f90 0x7ff588157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.665+0000 7ff58659c640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5880b7750 0x7ff58815acb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c400 tx=0x7ff570001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.665+0000 7ff57f7fe640 1 -- 172.21.15.66:0/1714096270 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570031040 con 0x7ff5880b7750 2024-09-10T22:35:32.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.665+0000 7ff57f7fe640 1 -- 172.21.15.66:0/1714096270 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff570003610 con 0x7ff5880b7750 2024-09-10T22:35:32.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.665+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5880b7ff0 con 0x7ff5880b7750 2024-09-10T22:35:32.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.665+0000 7ff57f7fe640 1 -- 172.21.15.66:0/1714096270 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570003a00 con 0x7ff5880b7750 2024-09-10T22:35:32.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.665+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5880b85b0 con 0x7ff5880b7750 2024-09-10T22:35:32.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.666+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5880ceab0 con 0x7ff5880b7750 2024-09-10T22:35:32.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.667+0000 7ff57f7fe640 1 -- 172.21.15.66:0/1714096270 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff570003ba0 con 0x7ff5880b7750 2024-09-10T22:35:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.667+0000 7ff57f7fe640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff568077fa0 0x7ff56807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.667+0000 7ff57f7fe640 1 -- 172.21.15.66:0/1714096270 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff5700ba020 con 0x7ff5880b7750 2024-09-10T22:35:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.667+0000 7ff586d9d640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff568077fa0 0x7ff56807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:32.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.668+0000 7ff586d9d640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff568077fa0 0x7ff56807a460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff58806c190 tx=0x7ff580006ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:32.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.671+0000 7ff57f7fe640 1 -- 172.21.15.66:0/1714096270 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff570082c00 con 0x7ff5880b7750 2024-09-10T22:35:32.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.891+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5880cb050 con 0x7ff5880b7750 2024-09-10T22:35:32.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.891+0000 7ff57f7fe640 1 -- 172.21.15.66:0/1714096270 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7ff570082500 con 0x7ff5880b7750 2024-09-10T22:35:32.894 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:32.894 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:32.894 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:32.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff568077fa0 msgr2=0x7ff56807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff568077fa0 0x7ff56807a460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff58806c190 tx=0x7ff580006ca0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5880b7750 msgr2=0x7ff58815acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:32.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5880b7750 0x7ff58815acb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c400 tx=0x7ff570001cf0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.896 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 shutdown_connections 2024-09-10T22:35:32.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff568077fa0 0x7ff56807a460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff588156f90 0x7ff588157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5880b7750 0x7ff58815acb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 --2- 172.21.15.66:0/1714096270 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff574004800 0x7ff58815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:32.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.893+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 >> 172.21.15.66:0/1714096270 conn(0x7ff5880d3f10 msgr2=0x7ff5880b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:32.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.894+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 shutdown_connections 2024-09-10T22:35:32.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:32.894+0000 7ff58d239640 1 -- 172.21.15.66:0/1714096270 wait complete. 2024-09-10T22:35:32.907 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:33.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.009+0000 7f1626117640 1 -- 172.21.15.66:0/1068818772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f162015f290 msgr2=0x7f16200cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.009+0000 7f1626117640 1 --2- 172.21.15.66:0/1068818772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f162015f290 0x7f16200cf8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16200b7dd0 tx=0x7f161002a440 comp rx=0 tx=0).stop 2024-09-10T22:35:33.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.010+0000 7f1626117640 1 -- 172.21.15.66:0/1068818772 shutdown_connections 2024-09-10T22:35:33.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.010+0000 7f1626117640 1 --2- 172.21.15.66:0/1068818772 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f16200cd910 0x7f16200ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.010+0000 7f1626117640 1 --2- 172.21.15.66:0/1068818772 >> v2:172.21.15.94:3300/0 conn(0x7f16200cfe30 0x7f16200cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.012 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.010+0000 7f1626117640 1 --2- 172.21.15.66:0/1068818772 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f162015f290 0x7f16200cf8f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.010+0000 7f1626117640 1 -- 172.21.15.66:0/1068818772 >> 172.21.15.66:0/1068818772 conn(0x7f16200d1d20 msgr2=0x7f16200d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:33.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.010+0000 7f1626117640 1 -- 172.21.15.66:0/1068818772 shutdown_connections 2024-09-10T22:35:33.013 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.010+0000 7f1626117640 1 -- 172.21.15.66:0/1068818772 wait complete. 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.011+0000 7f1626117640 1 Processor -- start 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.011+0000 7f1626117640 1 -- start start 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1626117640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16200cd910 0x7f1620154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1626117640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16200cfe30 0x7f16201550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1626117640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f162015f290 0x7f16201e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1626117640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f162015f7f0 con 0x7f16200cd910 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1626117640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f162015f670 con 0x7f16200cfe30 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1626117640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f16200ce3c0 con 0x7f162015f290 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16200cd910 0x7f1620154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16200cd910 0x7f1620154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46812/0 (socket says 172.21.15.66:46812) 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 -- 172.21.15.66:0/2686673696 learned_addr learned my addr 172.21.15.66:0/2686673696 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161ffff640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f162015f290 0x7f16201e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.015 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161effd640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16200cfe30 0x7f16201550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 -- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f162015f290 msgr2=0x7f16201e8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f162015f290 0x7f16201e8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 -- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16200cfe30 msgr2=0x7f16201550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16200cfe30 0x7f16201550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 -- 172.21.15.66:0/2686673696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162006fc40 con 0x7f16200cd910 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f161f7fe640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16200cd910 0x7f1620154900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c00 tx=0x7f16100039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1607fff640 1 -- 172.21.15.66:0/2686673696 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161003e030 con 0x7f16200cd910 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1607fff640 1 -- 172.21.15.66:0/2686673696 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16100047a0 con 0x7f16200cd910 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.012+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16201f4ba0 con 0x7f16200cd910 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.013+0000 7f1607fff640 1 -- 172.21.15.66:0/2686673696 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610035040 con 0x7f16200cd910 2024-09-10T22:35:33.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.013+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162006efb0 con 0x7f16200cd910 2024-09-10T22:35:33.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.013+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16200cd3d0 con 0x7f16200cd910 2024-09-10T22:35:33.017 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.014+0000 7f1607fff640 1 -- 172.21.15.66:0/2686673696 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1610004b70 con 0x7f16200cd910 2024-09-10T22:35:33.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.015+0000 7f1607fff640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f160c077f70 0x7f160c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.015+0000 7f1607fff640 1 -- 172.21.15.66:0/2686673696 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f16100ba7c0 con 0x7f16200cd910 2024-09-10T22:35:33.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.015+0000 7f161effd640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f160c077f70 0x7f160c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.016+0000 7f161effd640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f160c077f70 0x7f160c07a430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16200ca090 tx=0x7f1614005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:33.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.019+0000 7f1607fff640 1 -- 172.21.15.66:0/2686673696 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1610082c70 con 0x7f16200cd910 2024-09-10T22:35:33.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.206+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16200cdf90 con 0x7f16200cd910 2024-09-10T22:35:33.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.207+0000 7f1607fff640 1 -- 172.21.15.66:0/2686673696 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16641 (secure 0 0 0) 0x7f16100876a0 con 0x7f16200cd910 2024-09-10T22:35:33.209 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:33.209 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:04.052417+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.209+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f160c077f70 msgr2=0x7f160c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f160c077f70 0x7f160c07a430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16200ca090 tx=0x7f1614005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16200cd910 msgr2=0x7f1620154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16200cd910 0x7f1620154900 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1610000c00 tx=0x7f16100039e0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 shutdown_connections 2024-09-10T22:35:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f160c077f70 0x7f160c07a430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f162015f290 0x7f16201e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16200cfe30 0x7f16201550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 --2- 172.21.15.66:0/2686673696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f16200cd910 0x7f1620154900 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 >> 172.21.15.66:0/2686673696 conn(0x7f16200d1d20 msgr2=0x7f16200b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:33.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 shutdown_connections 2024-09-10T22:35:33.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.210+0000 7f1626117640 1 -- 172.21.15.66:0/2686673696 wait complete. 2024-09-10T22:35:33.223 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:35:33.223 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:33.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.326+0000 7f968bc40640 1 -- 172.21.15.66:0/1505511719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f966c003a50 msgr2=0x7f966c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.326+0000 7f968bc40640 1 --2- 172.21.15.66:0/1505511719 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f966c003a50 0x7f966c003e70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9674009aa0 tx=0x7f966c005830 comp rx=0 tx=0).stop 2024-09-10T22:35:33.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.326+0000 7f968a1dd640 1 -- 172.21.15.66:0/1505511719 reap_dead start 2024-09-10T22:35:33.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.326+0000 7f968bc40640 1 -- 172.21.15.66:0/1505511719 shutdown_connections 2024-09-10T22:35:33.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.326+0000 7f968bc40640 1 -- 172.21.15.66:0/1505511719 >> 172.21.15.66:0/1505511719 conn(0x7f96840d0430 msgr2=0x7f96840d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:33.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.326+0000 7f968bc40640 1 -- 172.21.15.66:0/1505511719 shutdown_connections 2024-09-10T22:35:33.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.326+0000 7f968bc40640 1 -- 172.21.15.66:0/1505511719 wait complete. 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 Processor -- start 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 -- start start 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f966c003a50 0x7f9684158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96841f93d0 0x7f96840c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96841f87c0 0x7f96841f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f96840b0c50 con 0x7f96841f93d0 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f96840b0ad0 con 0x7f96841f87c0 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.328+0000 7f968bc40640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f96840741d0 con 0x7f966c003a50 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96841f93d0 0x7f96840c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96841f93d0 0x7f96840c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46836/0 (socket says 172.21.15.66:46836) 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 -- 172.21.15.66:0/2034397391 learned_addr learned my addr 172.21.15.66:0/2034397391 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f968a1dd640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96841f87c0 0x7f96841f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96899dc640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f966c003a50 0x7f9684158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 -- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f966c003a50 msgr2=0x7f9684158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f966c003a50 0x7f9684158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 -- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96841f87c0 msgr2=0x7f96841f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96841f87c0 0x7f96841f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 -- 172.21.15.66:0/2034397391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96840b3da0 con 0x7f96841f93d0 2024-09-10T22:35:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f968a1dd640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96841f87c0 0x7f96841f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96899dc640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f966c003a50 0x7f9684158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96891db640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96841f93d0 0x7f96840c9f50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96780066b0 tx=0x7f96780039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96827fc640 1 -- 172.21.15.66:0/2034397391 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678015070 con 0x7f96841f93d0 2024-09-10T22:35:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96827fc640 1 -- 172.21.15.66:0/2034397391 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96780119e0 con 0x7f96841f93d0 2024-09-10T22:35:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96840b4080 con 0x7f96841f93d0 2024-09-10T22:35:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f96827fc640 1 -- 172.21.15.66:0/2034397391 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678011e30 con 0x7f96841f93d0 2024-09-10T22:35:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.329+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96840b4640 con 0x7f96841f93d0 2024-09-10T22:35:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.330+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96840746e0 con 0x7f96841f93d0 2024-09-10T22:35:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.331+0000 7f96827fc640 1 -- 172.21.15.66:0/2034397391 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f96780103c0 con 0x7f96841f93d0 2024-09-10T22:35:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.331+0000 7f96827fc640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9670077fa0 0x7f967007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.332+0000 7f96827fc640 1 -- 172.21.15.66:0/2034397391 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f967809ac90 con 0x7f96841f93d0 2024-09-10T22:35:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.332+0000 7f96899dc640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9670077fa0 0x7f967007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.332+0000 7f96899dc640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9670077fa0 0x7f967007a460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9674037000 tx=0x7f96740046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:33.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.336+0000 7f96827fc640 1 -- 172.21.15.66:0/2034397391 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9684074f20 con 0x7f96841f93d0 2024-09-10T22:35:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.562+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96840b1670 con 0x7f96841f93d0 2024-09-10T22:35:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.563+0000 7f96827fc640 1 -- 172.21.15.66:0/2034397391 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f9678063820 con 0x7f96841f93d0 2024-09-10T22:35:33.565 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:33.565 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:33.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9670077fa0 msgr2=0x7f967007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9670077fa0 0x7f967007a460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9674037000 tx=0x7f96740046c0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96841f93d0 msgr2=0x7f96840c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96841f93d0 0x7f96840c9f50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f96780066b0 tx=0x7f96780039c0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 shutdown_connections 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9670077fa0 0x7f967007a460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96841f87c0 0x7f96841f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f96841f93d0 0x7f96840c9f50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 --2- 172.21.15.66:0/2034397391 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f966c003a50 0x7f9684158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 >> 172.21.15.66:0/2034397391 conn(0x7f96840d0430 msgr2=0x7f968406fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 shutdown_connections 2024-09-10T22:35:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.565+0000 7f968bc40640 1 -- 172.21.15.66:0/2034397391 wait complete. 2024-09-10T22:35:33.578 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:33.684 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.682+0000 7f3999f2b640 1 -- 172.21.15.66:0/1978565816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3994154e90 msgr2=0x7f399406ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.682+0000 7f3999f2b640 1 --2- 172.21.15.66:0/1978565816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3994154e90 0x7f399406ecb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3984009860 tx=0x7f398402a530 comp rx=0 tx=0).stop 2024-09-10T22:35:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.682+0000 7f3999f2b640 1 -- 172.21.15.66:0/1978565816 shutdown_connections 2024-09-10T22:35:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.682+0000 7f3999f2b640 1 --2- 172.21.15.66:0/1978565816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3994154e90 0x7f399406ecb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.682+0000 7f3999f2b640 1 -- 172.21.15.66:0/1978565816 >> 172.21.15.66:0/1978565816 conn(0x7f39940b16d0 msgr2=0x7f39940b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.682+0000 7f3999f2b640 1 -- 172.21.15.66:0/1978565816 shutdown_connections 2024-09-10T22:35:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.682+0000 7f3999f2b640 1 -- 172.21.15.66:0/1978565816 wait complete. 2024-09-10T22:35:33.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.683+0000 7f3999f2b640 1 Processor -- start 2024-09-10T22:35:33.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.683+0000 7f3999f2b640 1 -- start start 2024-09-10T22:35:33.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3999f2b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39940cef70 0x7f39941f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3999f2b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3994154e90 0x7f39940c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3999f2b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3994155270 0x7f39941f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.687 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3999f2b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3994073570 con 0x7f3994155270 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3999f2b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f39940733f0 con 0x7f3994154e90 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3999f2b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f39940736f0 con 0x7f39940cef70 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3993fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3994155270 0x7f39941f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3993fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3994155270 0x7f39941f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38128/0 (socket says 172.21.15.66:38128) 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f3993fff640 1 -- 172.21.15.66:0/3325762825 learned_addr learned my addr 172.21.15.66:0/3325762825 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f398bfff640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3994154e90 0x7f39940c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.684+0000 7f39937fe640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39940cef70 0x7f39941f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3993fff640 1 -- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39940cef70 msgr2=0x7f39941f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3993fff640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39940cef70 0x7f39941f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3993fff640 1 -- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3994154e90 msgr2=0x7f39940c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3993fff640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3994154e90 0x7f39940c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3993fff640 1 -- 172.21.15.66:0/3325762825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39941f2350 con 0x7f3994155270 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3993fff640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3994155270 0x7f39941f1e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f398c00cb70 tx=0x7f398c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:33.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3990ff9640 1 -- 172.21.15.66:0/3325762825 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c016020 con 0x7f3994155270 2024-09-10T22:35:33.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3990ff9640 1 -- 172.21.15.66:0/3325762825 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f398c00ae50 con 0x7f3994155270 2024-09-10T22:35:33.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941542a0 con 0x7f3994155270 2024-09-10T22:35:33.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3990ff9640 1 -- 172.21.15.66:0/3325762825 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c012660 con 0x7f3994155270 2024-09-10T22:35:33.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.685+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39941f4c90 con 0x7f3994155270 2024-09-10T22:35:33.689 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.686+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3994073c20 con 0x7f3994155270 2024-09-10T22:35:33.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.687+0000 7f3990ff9640 1 -- 172.21.15.66:0/3325762825 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f398c01b400 con 0x7f3994155270 2024-09-10T22:35:33.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.687+0000 7f3990ff9640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3974077f70 0x7f397407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:33.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.688+0000 7f3990ff9640 1 -- 172.21.15.66:0/3325762825 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f398c09a660 con 0x7f3994155270 2024-09-10T22:35:33.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.688+0000 7f39937fe640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3974077f70 0x7f397407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:33.690 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.688+0000 7f39937fe640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3974077f70 0x7f397407a430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3984034000 tx=0x7f3984004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:33.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.692+0000 7f3990ff9640 1 -- 172.21.15.66:0/3325762825 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f398c020020 con 0x7f3994155270 2024-09-10T22:35:33.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.918+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39940ccc90 con 0x7f3994155270 2024-09-10T22:35:33.920 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.918+0000 7f3990ff9640 1 -- 172.21.15.66:0/3325762825 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5214 (secure 0 0 0) 0x7f398c062b10 con 0x7f3994155270 2024-09-10T22:35:33.921 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:33.921 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"btime":"2024-09-10T22:35:06:064315+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4292,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2636553491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":2636553491},{"type":"v1","addr":"172.21.15.94:6837","nonce":2636553491}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4351,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3592320280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3592320280},{"type":"v1","addr":"172.21.15.66:6835","nonce":3592320280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/3162194556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3162194556},{"type":"v1","addr":"172.21.15.94:6836","nonce":3162194556}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:02.418856+0000","modified":"2024-09-10T22:35:05.440441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6834/695657263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":695657263},{"type":"v1","addr":"172.21.15.66:6834","nonce":695657263}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T22:35:33.921 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2024-09-10T22:35:33.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3974077f70 msgr2=0x7f397407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3974077f70 0x7f397407a430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3984034000 tx=0x7f3984004730 comp rx=0 tx=0).stop 2024-09-10T22:35:33.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3994155270 msgr2=0x7f39941f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:33.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3994155270 0x7f39941f1e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f398c00cb70 tx=0x7f398c00a430 comp rx=0 tx=0).stop 2024-09-10T22:35:33.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 shutdown_connections 2024-09-10T22:35:33.923 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3974077f70 0x7f397407a430 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3994155270 0x7f39941f1e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3994154e90 0x7f39940c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 --2- 172.21.15.66:0/3325762825 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f39940cef70 0x7f39941f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:33.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 >> 172.21.15.66:0/3325762825 conn(0x7f39940b16d0 msgr2=0x7f39940cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:33.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 shutdown_connections 2024-09-10T22:35:33.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:33.921+0000 7f3999f2b640 1 -- 172.21.15.66:0/3325762825 wait complete. 2024-09-10T22:35:33.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:34.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 -- 172.21.15.66:0/717048560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15a8e0 msgr2=0x7f33fc0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 --2- 172.21.15.66:0/717048560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15a8e0 0x7f33fc0ce860 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0b6dd0 tx=0x7f33ec02a5f0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 -- 172.21.15.66:0/717048560 shutdown_connections 2024-09-10T22:35:34.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 --2- 172.21.15.66:0/717048560 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33fc15acc0 0x7f33fc0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 --2- 172.21.15.66:0/717048560 >> v2:172.21.15.94:3300/0 conn(0x7f33fc0cc1b0 0x7f33fc0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 --2- 172.21.15.66:0/717048560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15a8e0 0x7f33fc0ce860 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.040 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 -- 172.21.15.66:0/717048560 >> 172.21.15.66:0/717048560 conn(0x7f33fc0d0c90 msgr2=0x7f33fc0d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:34.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 -- 172.21.15.66:0/717048560 shutdown_connections 2024-09-10T22:35:34.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.037+0000 7f3402052640 1 -- 172.21.15.66:0/717048560 wait complete. 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.038+0000 7f3402052640 1 Processor -- start 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.039+0000 7f3402052640 1 -- start start 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.039+0000 7f3402052640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33fc15a8e0 0x7f33fc152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.039+0000 7f3402052640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15acc0 0x7f33fc0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.039+0000 7f3402052640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33fc0c94d0 0x7f33fc15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.039+0000 7f3402052640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc070a20 con 0x7f33fc15acc0 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.039+0000 7f3402052640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc070ba0 con 0x7f33fc0c94d0 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.039+0000 7f3402052640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f33fc070d20 con 0x7f33fc15a8e0 2024-09-10T22:35:34.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15acc0 0x7f33fc0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15acc0 0x7f33fc0c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38156/0 (socket says 172.21.15.66:38156) 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 -- 172.21.15.66:0/1496954959 learned_addr learned my addr 172.21.15.66:0/1496954959 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33fb7fe640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33fc15a8e0 0x7f33fc152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33fbfff640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33fc0c94d0 0x7f33fc15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 -- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33fc15a8e0 msgr2=0x7f33fc152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33fc15a8e0 0x7f33fc152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 -- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33fc0c94d0 msgr2=0x7f33fc15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33fc0c94d0 0x7f33fc15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 -- 172.21.15.66:0/1496954959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc1537b0 con 0x7f33fc15acc0 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33fb7fe640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33fc15a8e0 0x7f33fc152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33faffd640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15acc0 0x7f33fc0c8f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f33f000d980 tx=0x7f33f000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33e3fff640 1 -- 172.21.15.66:0/1496954959 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0017070 con 0x7f33fc15acc0 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33e3fff640 1 -- 172.21.15.66:0/1496954959 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33f0009cc0 con 0x7f33fc15acc0 2024-09-10T22:35:34.045 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc153a90 con 0x7f33fc15acc0 2024-09-10T22:35:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f33e3fff640 1 -- 172.21.15.66:0/1496954959 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f00106c0 con 0x7f33fc15acc0 2024-09-10T22:35:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.040+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1f5ed0 con 0x7f33fc15acc0 2024-09-10T22:35:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.041+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33fc0ce860 con 0x7f33fc15acc0 2024-09-10T22:35:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.042+0000 7f33e3fff640 1 -- 172.21.15.66:0/1496954959 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f33f0010860 con 0x7f33fc15acc0 2024-09-10T22:35:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.042+0000 7f33e3fff640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f33e8077f70 0x7f33e807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.042+0000 7f33e3fff640 1 -- 172.21.15.66:0/1496954959 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f33f009a050 con 0x7f33fc15acc0 2024-09-10T22:35:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.043+0000 7f33fb7fe640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f33e8077f70 0x7f33e807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.043+0000 7f33fb7fe640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f33e8077f70 0x7f33e807a430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33ec033000 tx=0x7f33ec004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.046+0000 7f33e3fff640 1 -- 172.21.15.66:0/1496954959 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f33f0062500 con 0x7f33fc15acc0 2024-09-10T22:35:34.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.236+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33fc074720 con 0x7f33fc15acc0 2024-09-10T22:35:34.239 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.237+0000 7f33e3fff640 1 -- 172.21.15.66:0/1496954959 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16641 (secure 0 0 0) 0x7f33f0066f30 con 0x7f33fc15acc0 2024-09-10T22:35:34.239 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:34.240 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":21,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:04.052417+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:00.287286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:01.380372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f33e8077f70 msgr2=0x7f33e807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f33e8077f70 0x7f33e807a430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33ec033000 tx=0x7f33ec004730 comp rx=0 tx=0).stop 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15acc0 msgr2=0x7f33fc0c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15acc0 0x7f33fc0c8f90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f33f000d980 tx=0x7f33f000de40 comp rx=0 tx=0).stop 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 shutdown_connections 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f33e8077f70 0x7f33e807a430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33fc0c94d0 0x7f33fc15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33fc15acc0 0x7f33fc0c8f90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 --2- 172.21.15.66:0/1496954959 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f33fc15a8e0 0x7f33fc152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.242 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 >> 172.21.15.66:0/1496954959 conn(0x7f33fc0d0c90 msgr2=0x7f33fc0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:34.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 shutdown_connections 2024-09-10T22:35:34.243 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.240+0000 7f3402052640 1 -- 172.21.15.66:0/1496954959 wait complete. 2024-09-10T22:35:34.253 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:35:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.358+0000 7ff1503d3640 1 -- 172.21.15.66:0/154124443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 msgr2=0x7ff130003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.358+0000 7ff1503d3640 1 --2- 172.21.15.66:0/154124443 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 0x7ff130003e50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff14000bbe0 tx=0x7ff130005830 comp rx=0 tx=0).stop 2024-09-10T22:35:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.358+0000 7ff14e970640 1 -- 172.21.15.66:0/154124443 reap_dead start 2024-09-10T22:35:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.358+0000 7ff1503d3640 1 -- 172.21.15.66:0/154124443 shutdown_connections 2024-09-10T22:35:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.358+0000 7ff1503d3640 1 -- 172.21.15.66:0/154124443 >> 172.21.15.66:0/154124443 conn(0x7ff1480d1d60 msgr2=0x7ff1480d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.358+0000 7ff1503d3640 1 -- 172.21.15.66:0/154124443 shutdown_connections 2024-09-10T22:35:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.358+0000 7ff1503d3640 1 -- 172.21.15.66:0/154124443 wait complete. 2024-09-10T22:35:34.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.359+0000 7ff1503d3640 1 Processor -- start 2024-09-10T22:35:34.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.359+0000 7ff1503d3640 1 -- start start 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff1503d3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 0x7ff14815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff1503d3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff14806f850 0x7ff14815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff1503d3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff14815be40 0x7ff14815d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff1503d3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481607a0 con 0x7ff130003a30 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff1503d3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff148160620 con 0x7ff14806f850 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff1503d3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff148160920 con 0x7ff14815be40 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 0x7ff14815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 0x7ff14815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38180/0 (socket says 172.21.15.66:38180) 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 -- 172.21.15.66:0/3507095118 learned_addr learned my addr 172.21.15.66:0/3507095118 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14d96e640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff14806f850 0x7ff14815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e970640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff14815be40 0x7ff14815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 -- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff14815be40 msgr2=0x7ff14815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff14815be40 0x7ff14815d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 -- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff14806f850 msgr2=0x7ff14815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff14806f850 0x7ff14815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 -- 172.21.15.66:0/3507095118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1480ce640 con 0x7ff130003a30 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14d96e640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff14806f850 0x7ff14815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.360+0000 7ff14e16f640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 0x7ff14815b1f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff14000bb00 tx=0x7ff1400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.361+0000 7ff146ffd640 1 -- 172.21.15.66:0/3507095118 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14003e070 con 0x7ff130003a30 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.361+0000 7ff146ffd640 1 -- 172.21.15.66:0/3507095118 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff140031470 con 0x7ff130003a30 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.361+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1480ce920 con 0x7ff130003a30 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.361+0000 7ff146ffd640 1 -- 172.21.15.66:0/3507095118 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400318c0 con 0x7ff130003a30 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.361+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1480ceee0 con 0x7ff130003a30 2024-09-10T22:35:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.362+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14806fc30 con 0x7ff130003a30 2024-09-10T22:35:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.362+0000 7ff146ffd640 1 -- 172.21.15.66:0/3507095118 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff14004b030 con 0x7ff130003a30 2024-09-10T22:35:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.363+0000 7ff146ffd640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff134077fa0 0x7ff13407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.363+0000 7ff146ffd640 1 -- 172.21.15.66:0/3507095118 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff1400bad10 con 0x7ff130003a30 2024-09-10T22:35:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.363+0000 7ff14d96e640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff134077fa0 0x7ff13407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.364+0000 7ff14d96e640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff134077fa0 0x7ff13407a460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff13c000fb0 tx=0x7ff13c0067a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:34.369 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.367+0000 7ff146ffd640 1 -- 172.21.15.66:0/3507095118 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff140083810 con 0x7ff130003a30 2024-09-10T22:35:34.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.589+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff14815c220 con 0x7ff130003a30 2024-09-10T22:35:34.592 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:35:34.590+0000 7fe21e4b6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:35:34.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.593+0000 7ff146ffd640 1 -- 172.21.15.66:0/3507095118 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7ff140083110 con 0x7ff130003a30 2024-09-10T22:35:34.595 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:35:34.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff134077fa0 msgr2=0x7ff13407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff134077fa0 0x7ff13407a460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff13c000fb0 tx=0x7ff13c0067a0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 msgr2=0x7ff14815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 0x7ff14815b1f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff14000bb00 tx=0x7ff1400038a0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 shutdown_connections 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff134077fa0 0x7ff13407a460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff14815be40 0x7ff14815d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff14806f850 0x7ff14815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 --2- 172.21.15.66:0/3507095118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff130003a30 0x7ff14815b1f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 >> 172.21.15.66:0/3507095118 conn(0x7ff1480d1d60 msgr2=0x7ff1480713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 shutdown_connections 2024-09-10T22:35:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.596+0000 7ff1503d3640 1 -- 172.21.15.66:0/3507095118 wait complete. 2024-09-10T22:35:34.602 INFO:tasks.ceph.mds.c.smithi066.stderr: -6900> 2024-09-10T22:35:08.027+0000 7f12bfdea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:35:34.610 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:35:34.634 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:35:34.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.716+0000 7f009e553640 1 -- 172.21.15.66:0/411415946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009815e120 msgr2=0x7f00980d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.716+0000 7f009e553640 1 --2- 172.21.15.66:0/411415946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009815e120 0x7f00980d4a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0098069870 tx=0x7f008802a440 comp rx=0 tx=0).stop 2024-09-10T22:35:34.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.716+0000 7f009e553640 1 -- 172.21.15.66:0/411415946 shutdown_connections 2024-09-10T22:35:34.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.716+0000 7f009e553640 1 --2- 172.21.15.66:0/411415946 >> v2:172.21.15.94:3300/0 conn(0x7f00980d1f90 0x7f00980d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.716+0000 7f009e553640 1 --2- 172.21.15.66:0/411415946 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f00980d10d0 0x7f00980cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.716+0000 7f009e553640 1 --2- 172.21.15.66:0/411415946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f009815e120 0x7f00980d4a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.717+0000 7f009e553640 1 -- 172.21.15.66:0/411415946 >> 172.21.15.66:0/411415946 conn(0x7f00980d3f10 msgr2=0x7f00980d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:34.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.717+0000 7f009e553640 1 -- 172.21.15.66:0/411415946 shutdown_connections 2024-09-10T22:35:34.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.717+0000 7f009e553640 1 -- 172.21.15.66:0/411415946 wait complete. 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 Processor -- start 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 -- start start 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00980d10d0 0x7f00981f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00980d1f90 0x7f00980a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f009815e120 0x7f009809dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f00980b6ee0 con 0x7f00980d1f90 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00980b6d60 con 0x7f00980d10d0 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.718+0000 7f009e553640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f00980b7060 con 0x7f009815e120 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00980d1f90 0x7f00980a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f0097fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00980d10d0 0x7f00981f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f009caf0640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f009815e120 0x7f009809dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00980d1f90 0x7f00980a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38216/0 (socket says 172.21.15.66:38216) 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 -- 172.21.15.66:0/1932461403 learned_addr learned my addr 172.21.15.66:0/1932461403 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 -- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f009815e120 msgr2=0x7f009809dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f009815e120 0x7f009809dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 -- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00980d10d0 msgr2=0x7f00981f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00980d10d0 0x7f00981f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 -- 172.21.15.66:0/1932461403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00981f5420 con 0x7f00980d1f90 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f008ffff640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00980d1f90 0x7f00980a1480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f008000d780 tx=0x7f008000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f00957fa640 1 -- 172.21.15.66:0/1932461403 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080017070 con 0x7f00980d1f90 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f00957fa640 1 -- 172.21.15.66:0/1932461403 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f008000de90 con 0x7f00980d1f90 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00981f48a0 con 0x7f00980d1f90 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f00957fa640 1 -- 172.21.15.66:0/1932461403 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080010460 con 0x7f00980d1f90 2024-09-10T22:35:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.719+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00981f5140 con 0x7f00980d1f90 2024-09-10T22:35:34.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.720+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00980b7de0 con 0x7f00980d1f90 2024-09-10T22:35:34.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.721+0000 7f00957fa640 1 -- 172.21.15.66:0/1932461403 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0080010600 con 0x7f00980d1f90 2024-09-10T22:35:34.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.722+0000 7f00957fa640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0078077f70 0x7f007807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:34.724 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.722+0000 7f00957fa640 1 -- 172.21.15.66:0/1932461403 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f008009a120 con 0x7f00980d1f90 2024-09-10T22:35:34.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.722+0000 7f0097fff640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0078077f70 0x7f007807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:34.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.722+0000 7f0097fff640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0078077f70 0x7f007807a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0088033000 tx=0x7f0088004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:34.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.726+0000 7f00957fa640 1 -- 172.21.15.66:0/1932461403 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0080062570 con 0x7f00980d1f90 2024-09-10T22:35:34.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:34.951+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f00981f8930 con 0x7f00980d1f90 2024-09-10T22:35:35.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.595+0000 7f00957fa640 1 -- 172.21.15.66:0/1932461403 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f0080066fa0 con 0x7f00980d1f90 2024-09-10T22:35:35.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0078077f70 msgr2=0x7f007807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:35.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0078077f70 0x7f007807a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0088033000 tx=0x7f0088004730 comp rx=0 tx=0).stop 2024-09-10T22:35:35.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00980d1f90 msgr2=0x7f00980a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:35.601 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00980d1f90 0x7f00980a1480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f008000d780 tx=0x7f008000dc40 comp rx=0 tx=0).stop 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 shutdown_connections 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0078077f70 0x7f007807a430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f009815e120 0x7f009809dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f00980d1f90 0x7f00980a1480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 --2- 172.21.15.66:0/1932461403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00980d10d0 0x7f00981f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 >> 172.21.15.66:0/1932461403 conn(0x7f00980d3f10 msgr2=0x7f00981612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 shutdown_connections 2024-09-10T22:35:35.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.599+0000 7f009e553640 1 -- 172.21.15.66:0/1932461403 wait complete. 2024-09-10T22:35:35.615 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:35:35.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/3462126924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e080ace80 msgr2=0x7f7e08070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:35.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/3462126924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e080ace80 0x7f7e08070230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009860 tx=0x7f7dfc02a4b0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/3462126924 shutdown_connections 2024-09-10T22:35:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/3462126924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e080ace80 0x7f7e08070230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/3462126924 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e0815bce0 0x7f7e0806cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/3462126924 >> 172.21.15.66:0/3462126924 conn(0x7f7e080b1c10 msgr2=0x7f7e080b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/3462126924 shutdown_connections 2024-09-10T22:35:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.769+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/3462126924 wait complete. 2024-09-10T22:35:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.770+0000 7f7e0e1e2640 1 Processor -- start 2024-09-10T22:35:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.770+0000 7f7e0e1e2640 1 -- start start 2024-09-10T22:35:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e0e1e2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e080ace80 0x7f7e0809d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e0e1e2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e0815b900 0x7f7e08156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e0e1e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0815bce0 0x7f7e08156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e0e1e2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7e080aed00 con 0x7f7e0815b900 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e0e1e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7e080aeb80 con 0x7f7e0815bce0 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e0e1e2640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7e080aee80 con 0x7f7e080ace80 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e06ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e0815b900 0x7f7e08156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e06ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e0815b900 0x7f7e08156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38232/0 (socket says 172.21.15.66:38232) 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.771+0000 7f7e06ffd640 1 -- 172.21.15.66:0/1444624096 learned_addr learned my addr 172.21.15.66:0/1444624096 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e077fe640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e080ace80 0x7f7e0809d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e06ffd640 1 -- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e080ace80 msgr2=0x7f7e0809d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e06ffd640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e080ace80 0x7f7e0809d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e06ffd640 1 -- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0815bce0 msgr2=0x7f7e08156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e07fff640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0815bce0 0x7f7e08156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e06ffd640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0815bce0 0x7f7e08156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e06ffd640 1 -- 172.21.15.66:0/1444624096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0809c390 con 0x7f7e0815b900 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e077fe640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e080ace80 0x7f7e0809d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e07fff640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0815bce0 0x7f7e08156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e06ffd640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e0815b900 0x7f7e08156800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009780 tx=0x7f7dfc032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7deffff640 1 -- 172.21.15.66:0/1444624096 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc040070 con 0x7f7e0815b900 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7deffff640 1 -- 172.21.15.66:0/1444624096 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dfc005330 con 0x7f7e0815b900 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7deffff640 1 -- 172.21.15.66:0/1444624096 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0047a0 con 0x7f7e0815b900 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0809c670 con 0x7f7e0815b900 2024-09-10T22:35:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.772+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0809cc30 con 0x7f7e0815b900 2024-09-10T22:35:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.773+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e080af3a0 con 0x7f7e0815b900 2024-09-10T22:35:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.774+0000 7f7deffff640 1 -- 172.21.15.66:0/1444624096 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7dfc004b50 con 0x7f7e0815b900 2024-09-10T22:35:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.774+0000 7f7deffff640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7df4077f70 0x7f7df407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.775+0000 7f7deffff640 1 -- 172.21.15.66:0/1444624096 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f7dfc0bb450 con 0x7f7e0815b900 2024-09-10T22:35:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.775+0000 7f7e077fe640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7df4077f70 0x7f7df407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.775+0000 7f7e077fe640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7df4077f70 0x7f7df407a430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7df8000f60 tx=0x7f7df8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:35.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.779+0000 7f7deffff640 1 -- 172.21.15.66:0/1444624096 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7dfc083f00 con 0x7f7e0815b900 2024-09-10T22:35:35.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:35.964+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7e0806eab0 con 0x7f7e0815b900 2024-09-10T22:35:36.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.601+0000 7f7deffff640 1 -- 172.21.15.66:0/1444624096 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f7dfc083800 con 0x7f7e0815b900 2024-09-10T22:35:36.603 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:35:36.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7df4077f70 msgr2=0x7f7df407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:36.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7df4077f70 0x7f7df407a430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7df8000f60 tx=0x7f7df8009040 comp rx=0 tx=0).stop 2024-09-10T22:35:36.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e0815b900 msgr2=0x7f7e08156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:36.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e0815b900 0x7f7e08156800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009780 tx=0x7f7dfc032c10 comp rx=0 tx=0).stop 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 shutdown_connections 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7df4077f70 0x7f7df407a430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e0815bce0 0x7f7e08156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e0815b900 0x7f7e08156800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 --2- 172.21.15.66:0/1444624096 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7e080ace80 0x7f7e0809d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 >> 172.21.15.66:0/1444624096 conn(0x7f7e080b1c10 msgr2=0x7f7e0806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 shutdown_connections 2024-09-10T22:35:36.607 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.604+0000 7f7e0e1e2640 1 -- 172.21.15.66:0/1444624096 wait complete. 2024-09-10T22:35:36.620 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:35:36.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 -- 172.21.15.66:0/3013264603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 msgr2=0x7f3e60162060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:36.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 --2- 172.21.15.66:0/3013264603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 0x7f3e60162060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e50005fd0 tx=0x7f3e5002a5d0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 -- 172.21.15.66:0/3013264603 shutdown_connections 2024-09-10T22:35:36.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 --2- 172.21.15.66:0/3013264603 >> v2:172.21.15.94:3300/0 conn(0x7f3e600d2aa0 0x7f3e600b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 --2- 172.21.15.66:0/3013264603 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 0x7f3e60162060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 -- 172.21.15.66:0/3013264603 >> 172.21.15.66:0/3013264603 conn(0x7f3e600d3f70 msgr2=0x7f3e600d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:36.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 -- 172.21.15.66:0/3013264603 shutdown_connections 2024-09-10T22:35:36.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.730+0000 7f3e65c33640 1 -- 172.21.15.66:0/3013264603 wait complete. 2024-09-10T22:35:36.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.731+0000 7f3e65c33640 1 Processor -- start 2024-09-10T22:35:36.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.731+0000 7f3e65c33640 1 -- start start 2024-09-10T22:35:36.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e65c33640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e600cfc00 0x7f3e601f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:36.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e65c33640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 0x7f3e60149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:36.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e65c33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e600d2aa0 0x7f3e600ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:36.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e65c33640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3e600b2d60 con 0x7f3e600cfc00 2024-09-10T22:35:36.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e65c33640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e600d4ce0 con 0x7f3e600d2aa0 2024-09-10T22:35:36.735 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e65c33640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3e600b2ee0 con 0x7f3e600d26c0 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e600cfc00 0x7f3e601f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e600cfc00 0x7f3e601f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38234/0 (socket says 172.21.15.66:38234) 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 -- 172.21.15.66:0/631345721 learned_addr learned my addr 172.21.15.66:0/631345721 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5effd640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 0x7f3e60149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5ffff640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e600d2aa0 0x7f3e600ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 -- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 msgr2=0x7f3e60149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 0x7f3e60149af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 -- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e600d2aa0 msgr2=0x7f3e600ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e600d2aa0 0x7f3e600ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5f7fe640 1 -- 172.21.15.66:0/631345721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6014a340 con 0x7f3e600cfc00 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.732+0000 7f3e5effd640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 0x7f3e60149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.733+0000 7f3e5f7fe640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e600cfc00 0x7f3e601f8c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009850 tx=0x7f3e500035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:36.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.733+0000 7f3e47fff640 1 -- 172.21.15.66:0/631345721 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5003f070 con 0x7f3e600cfc00 2024-09-10T22:35:36.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.733+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6014a620 con 0x7f3e600cfc00 2024-09-10T22:35:36.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.733+0000 7f3e47fff640 1 -- 172.21.15.66:0/631345721 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e50003ce0 con 0x7f3e600cfc00 2024-09-10T22:35:36.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.733+0000 7f3e47fff640 1 -- 172.21.15.66:0/631345721 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50035030 con 0x7f3e600cfc00 2024-09-10T22:35:36.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.733+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e601f4eb0 con 0x7f3e600cfc00 2024-09-10T22:35:36.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.734+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e600b0c40 con 0x7f3e600cfc00 2024-09-10T22:35:36.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.734+0000 7f3e47fff640 1 -- 172.21.15.66:0/631345721 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3e50004a00 con 0x7f3e600cfc00 2024-09-10T22:35:36.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.735+0000 7f3e47fff640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3e4c077f70 0x7f3e4c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:36.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.735+0000 7f3e47fff640 1 -- 172.21.15.66:0/631345721 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7f3e500bab90 con 0x7f3e600cfc00 2024-09-10T22:35:36.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.735+0000 7f3e5effd640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3e4c077f70 0x7f3e4c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:36.738 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.735+0000 7f3e5effd640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3e4c077f70 0x7f3e4c07a430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e6015ea60 tx=0x7f3e54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:36.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.739+0000 7f3e47fff640 1 -- 172.21.15.66:0/631345721 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3e50083770 con 0x7f3e600cfc00 2024-09-10T22:35:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:36.927+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3e600b3ae0 con 0x7f3e600cfc00 2024-09-10T22:35:37.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.602+0000 7f3e47fff640 1 -- 172.21.15.66:0/631345721 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f3e50083070 con 0x7f3e600cfc00 2024-09-10T22:35:37.605 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3e4c077f70 msgr2=0x7f3e4c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3e4c077f70 0x7f3e4c07a430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e6015ea60 tx=0x7f3e54009040 comp rx=0 tx=0).stop 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e600cfc00 msgr2=0x7f3e601f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e600cfc00 0x7f3e601f8c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009850 tx=0x7f3e500035f0 comp rx=0 tx=0).stop 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 shutdown_connections 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3e4c077f70 0x7f3e4c07a430 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e6015ea60 tx=0x7f3e54009040 comp rx=0 tx=0).stop 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e600d2aa0 0x7f3e600ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:37.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3e600d26c0 0x7f3e60149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:37.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 --2- 172.21.15.66:0/631345721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e600cfc00 0x7f3e601f8c50 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009850 tx=0x7f3e500035f0 comp rx=0 tx=0).stop 2024-09-10T22:35:37.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 >> 172.21.15.66:0/631345721 conn(0x7f3e600d3f70 msgr2=0x7f3e600d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:37.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.606+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 shutdown_connections 2024-09-10T22:35:37.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.607+0000 7f3e65c33640 1 -- 172.21.15.66:0/631345721 wait complete. 2024-09-10T22:35:37.623 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:35:37.623 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:35:37.624 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:35:37.624 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:37.624 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:35:37.624 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:35:37.624 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:35:37.624 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:37.624 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:35:37.624 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:35:37.625 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:35:37.625 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:37.625 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:35:37.625 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:35:37.625 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:35:37.625 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:37.635 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:35:37.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:35:37.636 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:35:37.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:35:37.638 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:35:37.638 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:35:37.638 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:35:37.639 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:35:37.679 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:35:37.679 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:37.680 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:35:37.680 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:37.680 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:35:37.681 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:37.681 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:35:37.682 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:37.682 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:35:37.760 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:35:37.760 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:35:37.763 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:35:37.763 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:35:37.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.794+0000 7f0a3f233640 1 -- 172.21.15.66:0/1141026803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 msgr2=0x7f0a28003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.794+0000 7f0a3f233640 1 --2- 172.21.15.66:0/1141026803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 0x7f0a28003dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0017c0 tx=0x7f0a28005830 comp rx=0 tx=0).stop 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.794+0000 7f0a3d7d0640 1 -- 172.21.15.66:0/1141026803 reap_dead start 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.794+0000 7f0a3f233640 1 -- 172.21.15.66:0/1141026803 shutdown_connections 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.794+0000 7f0a3f233640 1 -- 172.21.15.66:0/1141026803 >> 172.21.15.66:0/1141026803 conn(0x7f0a380b1660 msgr2=0x7f0a380b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.794+0000 7f0a3f233640 1 -- 172.21.15.66:0/1141026803 shutdown_connections 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.794+0000 7f0a3f233640 1 -- 172.21.15.66:0/1141026803 wait complete. 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.796+0000 7f0a3f233640 1 Processor -- start 2024-09-10T22:35:37.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.796+0000 7f0a3f233640 1 -- start start 2024-09-10T22:35:37.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.796+0000 7f0a3f233640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 0x7f0a380c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:37.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.796+0000 7f0a3f233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a381550a0 0x7f0a381571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:37.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3f233640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a38155d70 0x7f0a3814bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:37.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3f233640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3815f710 con 0x7f0a280039e0 2024-09-10T22:35:37.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3f233640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3806f930 con 0x7f0a381550a0 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3f233640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3815f890 con 0x7f0a38155d70 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 0x7f0a380c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 0x7f0a380c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38276/0 (socket says 172.21.15.66:38276) 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 -- 172.21.15.66:0/1674830815 learned_addr learned my addr 172.21.15.66:0/1674830815 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3d7d0640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a38155d70 0x7f0a3814bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a37fff640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a381550a0 0x7f0a381571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 -- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a38155d70 msgr2=0x7f0a3814bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a38155d70 0x7f0a3814bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 -- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a381550a0 msgr2=0x7f0a381571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a381550a0 0x7f0a381571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 -- 172.21.15.66:0/1674830815 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a380ca710 con 0x7f0a280039e0 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3cfcf640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 0x7f0a380c9fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0018b0 tx=0x7f0a2c0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:37.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a357fa640 1 -- 172.21.15.66:0/1674830815 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c037040 con 0x7f0a280039e0 2024-09-10T22:35:37.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a357fa640 1 -- 172.21.15.66:0/1674830815 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a2c041070 con 0x7f0a280039e0 2024-09-10T22:35:37.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a380ca9f0 con 0x7f0a280039e0 2024-09-10T22:35:37.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a357fa640 1 -- 172.21.15.66:0/1674830815 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c003830 con 0x7f0a280039e0 2024-09-10T22:35:37.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.797+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a38156c80 con 0x7f0a280039e0 2024-09-10T22:35:37.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.798+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3806e510 con 0x7f0a280039e0 2024-09-10T22:35:37.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.799+0000 7f0a357fa640 1 -- 172.21.15.66:0/1674830815 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0a2c0039d0 con 0x7f0a280039e0 2024-09-10T22:35:37.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.799+0000 7f0a357fa640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a1c077f70 0x7f0a1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:37.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.800+0000 7f0a357fa640 1 -- 172.21.15.66:0/1674830815 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f0a2c0b9c70 con 0x7f0a280039e0 2024-09-10T22:35:37.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.800+0000 7f0a37fff640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a1c077f70 0x7f0a1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:37.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.800+0000 7f0a37fff640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a1c077f70 0x7f0a1c07a430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a381f5550 tx=0x7f0a24001e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:37.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:37.804+0000 7f0a357fa640 1 -- 172.21.15.66:0/1674830815 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0a2c082390 con 0x7f0a280039e0 2024-09-10T22:35:38.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.001+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0a3806e940 con 0x7f0a280039e0 2024-09-10T22:35:38.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.618+0000 7f0a357fa640 1 -- 172.21.15.66:0/1674830815 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f0a2c086dc0 con 0x7f0a280039e0 2024-09-10T22:35:38.620 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.621+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a1c077f70 msgr2=0x7f0a1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.621+0000 7f0a3f233640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a1c077f70 0x7f0a1c07a430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a381f5550 tx=0x7f0a24001e70 comp rx=0 tx=0).stop 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 msgr2=0x7f0a380c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 0x7f0a380c9fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0018b0 tx=0x7f0a2c0060e0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 shutdown_connections 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0a1c077f70 0x7f0a1c07a430 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0a38155d70 0x7f0a3814bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a381550a0 0x7f0a381571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.624 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 --2- 172.21.15.66:0/1674830815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0a280039e0 0x7f0a380c9fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 >> 172.21.15.66:0/1674830815 conn(0x7f0a380b1660 msgr2=0x7f0a380b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:38.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 shutdown_connections 2024-09-10T22:35:38.625 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.622+0000 7f0a3f233640 1 -- 172.21.15.66:0/1674830815 wait complete. 2024-09-10T22:35:38.637 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.742+0000 7fe976216640 1 -- 172.21.15.66:0/3705933590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 msgr2=0x7fe9700ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.742+0000 7fe976216640 1 --2- 172.21.15.66:0/3705933590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700ceab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe97008c310 tx=0x7fe96802aa30 comp rx=0 tx=0).stop 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.742+0000 7fe976216640 1 -- 172.21.15.66:0/3705933590 shutdown_connections 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.742+0000 7fe976216640 1 --2- 172.21.15.66:0/3705933590 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700ceab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.742+0000 7fe976216640 1 --2- 172.21.15.66:0/3705933590 >> v2:172.21.15.94:3300/0 conn(0x7fe9700d1ae0 0x7fe9700d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.742+0000 7fe976216640 1 --2- 172.21.15.66:0/3705933590 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe97015e090 0x7fe9700d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.742+0000 7fe976216640 1 -- 172.21.15.66:0/3705933590 >> 172.21.15.66:0/3705933590 conn(0x7fe9700d3f10 msgr2=0x7fe9700d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.743+0000 7fe976216640 1 -- 172.21.15.66:0/3705933590 shutdown_connections 2024-09-10T22:35:38.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.743+0000 7fe976216640 1 -- 172.21.15.66:0/3705933590 wait complete. 2024-09-10T22:35:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.744+0000 7fe976216640 1 Processor -- start 2024-09-10T22:35:38.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.744+0000 7fe976216640 1 -- start start 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9700d1ae0 0x7fe970157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe97015e090 0x7fe97006dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700cf380 con 0x7fe9700d2a10 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700cf200 con 0x7fe97015e090 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700cf500 con 0x7fe9700d1ae0 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38296/0 (socket says 172.21.15.66:38296) 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 -- 172.21.15.66:0/2610612155 learned_addr learned my addr 172.21.15.66:0/2610612155 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96f7fe640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9700d1ae0 0x7fe970157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96ffff640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe97015e090 0x7fe97006dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 -- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9700d1ae0 msgr2=0x7fe970157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9700d1ae0 0x7fe970157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 -- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe97015e090 msgr2=0x7fe97006dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe97015e090 0x7fe97006dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 -- 172.21.15.66:0/2610612155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe970157990 con 0x7fe9700d2a10 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe96effd640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700cb9d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe96400d770 tx=0x7fe96400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe957fff640 1 -- 172.21.15.66:0/2610612155 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964017070 con 0x7fe9700d2a10 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe957fff640 1 -- 172.21.15.66:0/2610612155 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe964009d20 con 0x7fe9700d2a10 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9700cc340 con 0x7fe9700d2a10 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe957fff640 1 -- 172.21.15.66:0/2610612155 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964004280 con 0x7fe9700d2a10 2024-09-10T22:35:38.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.745+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701f7fa0 con 0x7fe9700d2a10 2024-09-10T22:35:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.747+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe970074620 con 0x7fe9700d2a10 2024-09-10T22:35:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.747+0000 7fe957fff640 1 -- 172.21.15.66:0/2610612155 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe9640035f0 con 0x7fe9700d2a10 2024-09-10T22:35:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.747+0000 7fe957fff640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe95c077f70 0x7fe95c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.748+0000 7fe957fff640 1 -- 172.21.15.66:0/2610612155 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fe964099a80 con 0x7fe9700d2a10 2024-09-10T22:35:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.748+0000 7fe96f7fe640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe95c077f70 0x7fe95c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:38.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.748+0000 7fe96f7fe640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe95c077f70 0x7fe95c07a430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe96000bad0 tx=0x7fe96000b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:38.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:38.752+0000 7fe957fff640 1 -- 172.21.15.66:0/2610612155 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe970074620 con 0x7fe9700d2a10 2024-09-10T22:35:39.010 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.008+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe9700d1150 con 0x7fe9700d2a10 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.008+0000 7fe957fff640 1 -- 172.21.15.66:0/2610612155 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7fe970157990 con 0x7fe9700d2a10 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmOynNABAAKH0qcVEjgabxdgbcMzFenw==", 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.011 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBm92zKDBAAU/37cpMQbv3hwxpOOF7wqw==", 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.012 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmJyYSCBAAHOWAMRZJfptyuhaBDmMdHg==", 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:35:39.013 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBm9InwExAAOmJR0PSYaLknUYrjqdx8aQ==", 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBfyeBmCe8JNBAANHCDeqvquGRJDSdZEF/nKg==", 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.014 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBiyeBmAHFpNhAAZ/NEsvKX+rtDoru1DhdjMQ==", 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:35:39.015 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmyeBmVeTpGhAAtIciTFU74AgMICAgqvD0/g==", 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBpyeBm/01eLBAAeiodosiLdSg26EyvnT+OeA==", 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.016 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBuyeBmyrwOABAAx2o/hEASH4yqt4ZmQA68Wg==", 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBxyeBmIZEGAhAAd8+PqJWUZGZyB7BHmz60xw==", 2024-09-10T22:35:39.017 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB0yeBmvBX4IBAAH7DccJtEh0K9AYIrb/1GjA==", 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.018 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB4yeBmGD4nBRAAWQ51L9Bx7x6P0dwTRRFe8Q==", 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmyY04GBAAcxW03/29zpbpxXzE+JrsFA==", 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.019 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmtjE1IRAAO/ItVtJZ2xGcQj/LcTecMQ==", 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmasn2HRAAv24FRmz9CPJsGJ8DA1hOIQ==", 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmz6LSJhAASEiDnC8gX9owU8E46MLj/w==", 2024-09-10T22:35:39.021 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbyeBm8C1oHxAAPrYLLz62iPAQFG0SQ04DjA==", 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.022 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mds", 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB+yeBmbVuVDRAAafcbsvKe+99ThfW2u1Cdew==", 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.023 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB+yeBmdWWVDRAACoalPICFKsxYvnU865LD2g==", 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-osd", 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB+yeBmY22VDRAAf+CWpICtWEFPCtYd/+GX6g==", 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB+yeBm/3SVDRAALEMT93z4IIb35tFeX9G9Ag==", 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB+yeBmIH+VDRAAF9k8IutBTX0O6hRC3Fl5+Q==", 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.025 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB+yeBmUYeVDRAAgfpPLXE7BrymjXG77qQ6VQ==", 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbyeBm80AWORAA002Ap5bZLKASMXqSSM15dw==", 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:35:39.026 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbyeBmchE7NRAAGJXMPuxRS+9kbKAHx1KX4A==", 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:35:39.027 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe95c077f70 msgr2=0x7fe95c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe95c077f70 0x7fe95c07a430 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe96000bad0 tx=0x7fe96000b2e0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 msgr2=0x7fe9700cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700cb9d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe96400d770 tx=0x7fe96400dc30 comp rx=0 tx=0).stop 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 shutdown_connections 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe95c077f70 0x7fe95c07a430 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe97015e090 0x7fe97006dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9700d2a10 0x7fe9700cb9d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 --2- 172.21.15.66:0/2610612155 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9700d1ae0 0x7fe970157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 >> 172.21.15.66:0/2610612155 conn(0x7fe9700d3f10 msgr2=0x7fe9700b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 shutdown_connections 2024-09-10T22:35:39.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.011+0000 7fe976216640 1 -- 172.21.15.66:0/2610612155 wait complete. 2024-09-10T22:35:39.030 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 -- 172.21.15.66:0/4042395505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb800d10d0 msgr2=0x7ffb800cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 --2- 172.21.15.66:0/4042395505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb800d10d0 0x7ffb800cf850 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffb8008ca40 tx=0x7ffb7002a410 comp rx=0 tx=0).stop 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 -- 172.21.15.66:0/4042395505 shutdown_connections 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 --2- 172.21.15.66:0/4042395505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb800d10d0 0x7ffb800cf850 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 --2- 172.21.15.66:0/4042395505 >> v2:172.21.15.94:3300/0 conn(0x7ffb8015e120 0x7ffb800d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 -- 172.21.15.66:0/4042395505 >> 172.21.15.66:0/4042395505 conn(0x7ffb800d3f10 msgr2=0x7ffb800d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 -- 172.21.15.66:0/4042395505 shutdown_connections 2024-09-10T22:35:39.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.134+0000 7ffb87d94640 1 -- 172.21.15.66:0/4042395505 wait complete. 2024-09-10T22:35:39.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 Processor -- start 2024-09-10T22:35:39.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 -- start start 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb800d10d0 0x7ffb801f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb800d1f90 0x7ffb801f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb8015e120 0x7ffb801f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffb800b75f0 con 0x7ffb8015e120 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb800b7470 con 0x7ffb800d1f90 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb87d94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffb800b7770 con 0x7ffb800d10d0 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb86331640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb8015e120 0x7ffb801f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb86331640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb8015e120 0x7ffb801f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38306/0 (socket says 172.21.15.66:38306) 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.136+0000 7ffb86331640 1 -- 172.21.15.66:0/3615547514 learned_addr learned my addr 172.21.15.66:0/3615547514 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb85b30640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb800d10d0 0x7ffb801f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb86331640 1 -- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb800d10d0 msgr2=0x7ffb801f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb8532f640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb800d1f90 0x7ffb801f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb86331640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb800d10d0 0x7ffb801f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb86331640 1 -- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb800d1f90 msgr2=0x7ffb801f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb86331640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb800d1f90 0x7ffb801f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb86331640 1 -- 172.21.15.66:0/3615547514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb800c79f0 con 0x7ffb8015e120 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb86331640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb8015e120 0x7ffb801f4910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800cbc0 tx=0x7ffb7800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb767fc640 1 -- 172.21.15.66:0/3615547514 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78016020 con 0x7ffb8015e120 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb767fc640 1 -- 172.21.15.66:0/3615547514 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb7800ae30 con 0x7ffb8015e120 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb767fc640 1 -- 172.21.15.66:0/3615547514 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78012660 con 0x7ffb8015e120 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb800c7cd0 con 0x7ffb8015e120 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.137+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb800c93a0 con 0x7ffb8015e120 2024-09-10T22:35:39.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.138+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb800c7f10 con 0x7ffb8015e120 2024-09-10T22:35:39.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.139+0000 7ffb767fc640 1 -- 172.21.15.66:0/3615547514 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffb7801b400 con 0x7ffb8015e120 2024-09-10T22:35:39.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.139+0000 7ffb767fc640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb64077f70 0x7ffb6407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.139+0000 7ffb767fc640 1 -- 172.21.15.66:0/3615547514 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7ffb7809a370 con 0x7ffb8015e120 2024-09-10T22:35:39.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.140+0000 7ffb85b30640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb64077f70 0x7ffb6407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.140+0000 7ffb85b30640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb64077f70 0x7ffb6407a430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0058c0 tx=0x7ffb7c005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:39.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.144+0000 7ffb767fc640 1 -- 172.21.15.66:0/3615547514 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ffb78020020 con 0x7ffb8015e120 2024-09-10T22:35:39.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.399+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ffb800d26d0 con 0x7ffb8015e120 2024-09-10T22:35:39.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.401+0000 7ffb767fc640 1 -- 172.21.15.66:0/3615547514 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7ffb78063250 con 0x7ffb8015e120 2024-09-10T22:35:39.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb64077f70 msgr2=0x7ffb6407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb64077f70 0x7ffb6407a430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0058c0 tx=0x7ffb7c005850 comp rx=0 tx=0).stop 2024-09-10T22:35:39.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb8015e120 msgr2=0x7ffb801f4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb8015e120 0x7ffb801f4910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800cbc0 tx=0x7ffb7800a430 comp rx=0 tx=0).stop 2024-09-10T22:35:39.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 shutdown_connections 2024-09-10T22:35:39.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb64077f70 0x7ffb6407a430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb8015e120 0x7ffb801f4910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb800d1f90 0x7ffb801f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 --2- 172.21.15.66:0/3615547514 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb800d10d0 0x7ffb801f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 >> 172.21.15.66:0/3615547514 conn(0x7ffb800d3f10 msgr2=0x7ffb800b83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:39.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 shutdown_connections 2024-09-10T22:35:39.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.404+0000 7ffb87d94640 1 -- 172.21.15.66:0/3615547514 wait complete. 2024-09-10T22:35:39.417 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T22:35:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 -- 172.21.15.66:0/2975013888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e960 msgr2=0x7f9d9c15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 --2- 172.21.15.66:0/2975013888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e960 0x7f9d9c15f710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005fd0 tx=0x7f9d9402a980 comp rx=0 tx=0).stop 2024-09-10T22:35:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 -- 172.21.15.66:0/2975013888 shutdown_connections 2024-09-10T22:35:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 --2- 172.21.15.66:0/2975013888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e960 0x7f9d9c15f710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 --2- 172.21.15.66:0/2975013888 >> v2:172.21.15.94:3300/0 conn(0x7f9d9c06ff50 0x7f9d9c15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 -- 172.21.15.66:0/2975013888 >> 172.21.15.66:0/2975013888 conn(0x7f9d9c0b1660 msgr2=0x7f9d9c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 -- 172.21.15.66:0/2975013888 shutdown_connections 2024-09-10T22:35:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.521+0000 7f9da3223640 1 -- 172.21.15.66:0/2975013888 wait complete. 2024-09-10T22:35:39.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.522+0000 7f9da3223640 1 Processor -- start 2024-09-10T22:35:39.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.522+0000 7f9da3223640 1 -- start start 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.523+0000 7f9da3223640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e510 0x7f9d9c149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.523+0000 7f9da3223640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d9c06e960 0x7f9d9c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.523+0000 7f9da3223640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d9c06ff50 0x7f9d9c1e5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.523+0000 7f9da3223640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c15fc50 con 0x7f9d9c06ff50 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.523+0000 7f9da3223640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0745f0 con 0x7f9d9c06e960 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.523+0000 7f9da3223640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c15fdd0 con 0x7f9d9c06e510 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.523+0000 7f9da17c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d9c06ff50 0x7f9d9c1e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d9c06ff50 0x7f9d9c1e5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38320/0 (socket says 172.21.15.66:38320) 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 -- 172.21.15.66:0/4006620058 learned_addr learned my addr 172.21.15.66:0/4006620058 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da0fbf640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e510 0x7f9d9c149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9d9bfff640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d9c06e960 0x7f9d9c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 -- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e510 msgr2=0x7f9d9c149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e510 0x7f9d9c149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 -- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d9c06e960 msgr2=0x7f9d9c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d9c06e960 0x7f9d9c0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 -- 172.21.15.66:0/4006620058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c0cdf30 con 0x7f9d9c06ff50 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da17c0640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d9c06ff50 0x7f9d9c1e5890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009d90 tx=0x7f9d94003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9d997fa640 1 -- 172.21.15.66:0/4006620058 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94047070 con 0x7f9d9c06ff50 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9d997fa640 1 -- 172.21.15.66:0/4006620058 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d94004540 con 0x7f9d9c06ff50 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c0ce210 con 0x7f9d9c06ff50 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9d997fa640 1 -- 172.21.15.66:0/4006620058 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94034070 con 0x7f9d9c06ff50 2024-09-10T22:35:39.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.524+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c0ceab0 con 0x7f9d9c06ff50 2024-09-10T22:35:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.525+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c160280 con 0x7f9d9c06ff50 2024-09-10T22:35:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.526+0000 7f9d997fa640 1 -- 172.21.15.66:0/4006620058 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9d94041030 con 0x7f9d9c06ff50 2024-09-10T22:35:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.527+0000 7f9d997fa640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d80077fa0 0x7f9d8007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.527+0000 7f9d997fa640 1 -- 172.21.15.66:0/4006620058 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f9d940ba900 con 0x7f9d9c06ff50 2024-09-10T22:35:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.527+0000 7f9da0fbf640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d80077fa0 0x7f9d8007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.527+0000 7f9da0fbf640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d80077fa0 0x7f9d8007a460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009ac0 tx=0x7f9d90005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:39.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.531+0000 7f9d997fa640 1 -- 172.21.15.66:0/4006620058 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9d9403f070 con 0x7f9d9c06ff50 2024-09-10T22:35:39.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.788+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f9d9c0cec80 con 0x7f9d9c06ff50 2024-09-10T22:35:39.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.792+0000 7f9d997fa640 1 -- 172.21.15.66:0/4006620058 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f9d940837e0 con 0x7f9d9c06ff50 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d80077fa0 msgr2=0x7f9d8007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d80077fa0 0x7f9d8007a460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009ac0 tx=0x7f9d90005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d9c06ff50 msgr2=0x7f9d9c1e5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d9c06ff50 0x7f9d9c1e5890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009d90 tx=0x7f9d94003f20 comp rx=0 tx=0).stop 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 shutdown_connections 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d80077fa0 0x7f9d8007a460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d9c06ff50 0x7f9d9c1e5890 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d9c06e960 0x7f9d9c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 --2- 172.21.15.66:0/4006620058 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9d9c06e510 0x7f9d9c149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 >> 172.21.15.66:0/4006620058 conn(0x7f9d9c0b1660 msgr2=0x7f9d9c0b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 shutdown_connections 2024-09-10T22:35:39.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.794+0000 7f9da3223640 1 -- 172.21.15.66:0/4006620058 wait complete. 2024-09-10T22:35:39.807 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T22:35:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.910+0000 7fbc01182640 1 -- 172.21.15.66:0/3773645232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 msgr2=0x7fbbfc0d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.910+0000 7fbc01182640 1 --2- 172.21.15.66:0/3773645232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4a30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc069870 tx=0x7fbbec02a440 comp rx=0 tx=0).stop 2024-09-10T22:35:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.910+0000 7fbc01182640 1 -- 172.21.15.66:0/3773645232 shutdown_connections 2024-09-10T22:35:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.910+0000 7fbc01182640 1 --2- 172.21.15.66:0/3773645232 >> v2:172.21.15.94:3300/0 conn(0x7fbbfc0d1f90 0x7fbbfc0d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.913 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.910+0000 7fbc01182640 1 --2- 172.21.15.66:0/3773645232 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbfc0d10d0 0x7fbbfc0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.910+0000 7fbc01182640 1 --2- 172.21.15.66:0/3773645232 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4a30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.910+0000 7fbc01182640 1 -- 172.21.15.66:0/3773645232 >> 172.21.15.66:0/3773645232 conn(0x7fbbfc0d3f10 msgr2=0x7fbbfc0d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:39.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.911+0000 7fbc01182640 1 -- 172.21.15.66:0/3773645232 shutdown_connections 2024-09-10T22:35:39.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.911+0000 7fbc01182640 1 -- 172.21.15.66:0/3773645232 wait complete. 2024-09-10T22:35:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.912+0000 7fbc01182640 1 Processor -- start 2024-09-10T22:35:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.912+0000 7fbc01182640 1 -- start start 2024-09-10T22:35:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbc01182640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbfc0d10d0 0x7fbbfc0a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbc01182640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbfc0d1f90 0x7fbbfc1f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbc01182640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.916 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbc01182640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc0b6ee0 con 0x7fbbfc15e120 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbc01182640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc0b6d60 con 0x7fbbfc0d10d0 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbc01182640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbbfc0b7060 con 0x7fbbfc0d1f90 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38340/0 (socket says 172.21.15.66:38340) 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 -- 172.21.15.66:0/2283746580 learned_addr learned my addr 172.21.15.66:0/2283746580 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfad9d640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbfc0d10d0 0x7fbbfc0a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbf3fff640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbfc0d1f90 0x7fbbfc1f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 -- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbfc0d1f90 msgr2=0x7fbbfc1f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbfc0d1f90 0x7fbbfc1f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 -- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbfc0d10d0 msgr2=0x7fbbfc0a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbfc0d10d0 0x7fbbfc0a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 -- 172.21.15.66:0/2283746580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbfc0a13a0 con 0x7fbbfc15e120 2024-09-10T22:35:39.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.913+0000 7fbbfb59e640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4560 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400eaa0 tx=0x7fbbf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.914+0000 7fbbf37fe640 1 -- 172.21.15.66:0/2283746580 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400cdb0 con 0x7fbbfc15e120 2024-09-10T22:35:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.914+0000 7fbbf37fe640 1 -- 172.21.15.66:0/2283746580 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf4004c10 con 0x7fbbfc15e120 2024-09-10T22:35:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.914+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbfc0a16e0 con 0x7fbbfc15e120 2024-09-10T22:35:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.914+0000 7fbbf37fe640 1 -- 172.21.15.66:0/2283746580 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4005020 con 0x7fbbfc15e120 2024-09-10T22:35:39.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.914+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbfc1f5600 con 0x7fbbfc15e120 2024-09-10T22:35:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.915+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbfc0b7620 con 0x7fbbfc15e120 2024-09-10T22:35:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.915+0000 7fbbf37fe640 1 -- 172.21.15.66:0/2283746580 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbbf400f040 con 0x7fbbfc15e120 2024-09-10T22:35:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.916+0000 7fbbf37fe640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbbdc077f70 0x7fbbdc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.916+0000 7fbbf37fe640 1 -- 172.21.15.66:0/2283746580 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fbbf409a060 con 0x7fbbfc15e120 2024-09-10T22:35:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.916+0000 7fbbfad9d640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbbdc077f70 0x7fbbdc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:39.919 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.916+0000 7fbbfad9d640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbbdc077f70 0x7fbbdc07a430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbbec033000 tx=0x7fbbec004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:39.922 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:39.920+0000 7fbbf37fe640 1 -- 172.21.15.66:0/2283746580 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbbfc0a16e0 con 0x7fbbfc15e120 2024-09-10T22:35:40.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.174+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fbbfc0b7e70 con 0x7fbbfc15e120 2024-09-10T22:35:40.180 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.178+0000 7fbbf37fe640 1 -- 172.21.15.66:0/2283746580 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7fbbf4016030 con 0x7fbbfc15e120 2024-09-10T22:35:40.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbbdc077f70 msgr2=0x7fbbdc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbbdc077f70 0x7fbbdc07a430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbbec033000 tx=0x7fbbec004730 comp rx=0 tx=0).stop 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 msgr2=0x7fbbfc0d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4560 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400eaa0 tx=0x7fbbf400ef60 comp rx=0 tx=0).stop 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 shutdown_connections 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbbdc077f70 0x7fbbdc07a430 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbbfc15e120 0x7fbbfc0d4560 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbbfc0d1f90 0x7fbbfc1f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 --2- 172.21.15.66:0/2283746580 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbfc0d10d0 0x7fbbfc0a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 >> 172.21.15.66:0/2283746580 conn(0x7fbbfc0d3f10 msgr2=0x7fbbfc0ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 shutdown_connections 2024-09-10T22:35:40.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.180+0000 7fbc01182640 1 -- 172.21.15.66:0/2283746580 wait complete. 2024-09-10T22:35:40.193 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T22:35:40.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 -- 172.21.15.66:0/3830614564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074440 msgr2=0x7efc9c074820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 --2- 172.21.15.66:0/3830614564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074440 0x7efc9c074820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0b2ce0 tx=0x7efc9802a4d0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 -- 172.21.15.66:0/3830614564 shutdown_connections 2024-09-10T22:35:40.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 --2- 172.21.15.66:0/3830614564 >> v2:172.21.15.94:3300/0 conn(0x7efc9c0ce4a0 0x7efc9c0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 --2- 172.21.15.66:0/3830614564 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc9c074d60 0x7efc9c0cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 --2- 172.21.15.66:0/3830614564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074440 0x7efc9c074820 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 -- 172.21.15.66:0/3830614564 >> 172.21.15.66:0/3830614564 conn(0x7efc9c0b1f00 msgr2=0x7efc9c0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:40.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 -- 172.21.15.66:0/3830614564 shutdown_connections 2024-09-10T22:35:40.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.298+0000 7efca353d640 1 -- 172.21.15.66:0/3830614564 wait complete. 2024-09-10T22:35:40.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.299+0000 7efca353d640 1 Processor -- start 2024-09-10T22:35:40.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.299+0000 7efca353d640 1 -- start start 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca353d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc9c074440 0x7efc9c1f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca353d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074d60 0x7efc9c0c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca353d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc9c0ce4a0 0x7efc9c0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca353d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c15ef20 con 0x7efc9c074d60 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca353d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c15eda0 con 0x7efc9c074440 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca353d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efc9c15f0a0 con 0x7efc9c0ce4a0 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074d60 0x7efc9c0c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074d60 0x7efc9c0c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38360/0 (socket says 172.21.15.66:38360) 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 -- 172.21.15.66:0/1078309306 learned_addr learned my addr 172.21.15.66:0/1078309306 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca1ada640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc9c0ce4a0 0x7efc9c0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca12d9640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc9c074440 0x7efc9c1f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 -- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc9c0ce4a0 msgr2=0x7efc9c0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc9c0ce4a0 0x7efc9c0c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 -- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc9c074440 msgr2=0x7efc9c1f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc9c074440 0x7efc9c1f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 -- 172.21.15.66:0/1078309306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c157d80 con 0x7efc9c074d60 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.300+0000 7efca0ad8640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074d60 0x7efc9c0c61f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00d7a0 tx=0x7efc8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.301+0000 7efc91ffb640 1 -- 172.21.15.66:0/1078309306 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c017070 con 0x7efc9c074d60 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.301+0000 7efc91ffb640 1 -- 172.21.15.66:0/1078309306 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc8c009d20 con 0x7efc9c074d60 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.301+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c158060 con 0x7efc9c074d60 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.301+0000 7efc91ffb640 1 -- 172.21.15.66:0/1078309306 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0106c0 con 0x7efc9c074d60 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.301+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c158620 con 0x7efc9c074d60 2024-09-10T22:35:40.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.302+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9c0cdf60 con 0x7efc9c074d60 2024-09-10T22:35:40.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.302+0000 7efc91ffb640 1 -- 172.21.15.66:0/1078309306 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efc8c010860 con 0x7efc9c074d60 2024-09-10T22:35:40.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.303+0000 7efc91ffb640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efc80077f70 0x7efc8007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.303+0000 7efc91ffb640 1 -- 172.21.15.66:0/1078309306 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7efc8c099e40 con 0x7efc9c074d60 2024-09-10T22:35:40.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.303+0000 7efca12d9640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efc80077f70 0x7efc8007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.304+0000 7efca12d9640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efc80077f70 0x7efc8007a430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc98033000 tx=0x7efc98004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:40.309 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.307+0000 7efc91ffb640 1 -- 172.21.15.66:0/1078309306 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7efc8c062640 con 0x7efc9c074d60 2024-09-10T22:35:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.572+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7efc9c0ceb20 con 0x7efc9c074d60 2024-09-10T22:35:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.576+0000 7efc91ffb640 1 -- 172.21.15.66:0/1078309306 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7efc8c067070 con 0x7efc9c074d60 2024-09-10T22:35:40.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efc80077f70 msgr2=0x7efc8007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efc80077f70 0x7efc8007a430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc98033000 tx=0x7efc98004730 comp rx=0 tx=0).stop 2024-09-10T22:35:40.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074d60 msgr2=0x7efc9c0c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074d60 0x7efc9c0c61f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00d7a0 tx=0x7efc8c00dc60 comp rx=0 tx=0).stop 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 shutdown_connections 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efc80077f70 0x7efc8007a430 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efc9c0ce4a0 0x7efc9c0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc9c074d60 0x7efc9c0c61f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 --2- 172.21.15.66:0/1078309306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc9c074440 0x7efc9c1f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 >> 172.21.15.66:0/1078309306 conn(0x7efc9c0b1f00 msgr2=0x7efc9c06fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 shutdown_connections 2024-09-10T22:35:40.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.578+0000 7efca353d640 1 -- 172.21.15.66:0/1078309306 wait complete. 2024-09-10T22:35:40.591 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T22:35:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 -- 172.21.15.66:0/2590700221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b000ca6a0 msgr2=0x7f7b0015ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/2590700221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b000ca6a0 0x7f7b0015ddc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7b000b2720 tx=0x7f7af802b910 comp rx=0 tx=0).stop 2024-09-10T22:35:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 -- 172.21.15.66:0/2590700221 shutdown_connections 2024-09-10T22:35:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/2590700221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b000ca6a0 0x7f7b0015ddc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/2590700221 >> v2:172.21.15.94:3300/0 conn(0x7f7b0006e5f0 0x7f7b000ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/2590700221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7b0015a150 0x7f7b0006f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 -- 172.21.15.66:0/2590700221 >> 172.21.15.66:0/2590700221 conn(0x7f7b000b1660 msgr2=0x7f7b000b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 -- 172.21.15.66:0/2590700221 shutdown_connections 2024-09-10T22:35:40.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.697+0000 7f7b07fd3640 1 -- 172.21.15.66:0/2590700221 wait complete. 2024-09-10T22:35:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.698+0000 7f7b07fd3640 1 Processor -- start 2024-09-10T22:35:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.698+0000 7f7b07fd3640 1 -- start start 2024-09-10T22:35:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b07fd3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7b0006e5f0 0x7f7b001473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b07fd3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b000ca6a0 0x7f7b000d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b07fd3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0015a150 0x7f7b001f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b07fd3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7b000cc7a0 con 0x7f7b0015a150 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b07fd3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0006fb10 con 0x7f7b000ca6a0 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b07fd3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7b000cc920 con 0x7f7b0006e5f0 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0015a150 0x7f7b001f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0015a150 0x7f7b001f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38378/0 (socket says 172.21.15.66:38378) 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 -- 172.21.15.66:0/1429830816 learned_addr learned my addr 172.21.15.66:0/1429830816 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b05d6f640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7b0006e5f0 0x7f7b001473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b0556e640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b000ca6a0 0x7f7b000d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 -- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7b0006e5f0 msgr2=0x7f7b001473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7b0006e5f0 0x7f7b001473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 -- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b000ca6a0 msgr2=0x7f7b000d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b000ca6a0 0x7f7b000d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 -- 172.21.15.66:0/1429830816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b001f35c0 con 0x7f7b0015a150 2024-09-10T22:35:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.699+0000 7f7b06570640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0015a150 0x7f7b001f2660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7af800ad00 tx=0x7f7af800a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.700+0000 7f7af27fc640 1 -- 172.21.15.66:0/1429830816 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8035040 con 0x7f7b0015a150 2024-09-10T22:35:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.700+0000 7f7af27fc640 1 -- 172.21.15.66:0/1429830816 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7af8004240 con 0x7f7b0015a150 2024-09-10T22:35:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.700+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b001f3000 con 0x7f7b0015a150 2024-09-10T22:35:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.700+0000 7f7af27fc640 1 -- 172.21.15.66:0/1429830816 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8004650 con 0x7f7b0015a150 2024-09-10T22:35:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.700+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b00146eb0 con 0x7f7b0015a150 2024-09-10T22:35:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.701+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b000ca160 con 0x7f7b0015a150 2024-09-10T22:35:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.701+0000 7f7af27fc640 1 -- 172.21.15.66:0/1429830816 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7af804d440 con 0x7f7b0015a150 2024-09-10T22:35:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.702+0000 7f7af27fc640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ae4077f70 0x7f7ae407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.702+0000 7f7af27fc640 1 -- 172.21.15.66:0/1429830816 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f7af80baec0 con 0x7f7b0015a150 2024-09-10T22:35:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.702+0000 7f7b05d6f640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ae4077f70 0x7f7ae407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.703+0000 7f7b05d6f640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ae4077f70 0x7f7ae407a430 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7af4004510 tx=0x7f7af400b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.706+0000 7f7af27fc640 1 -- 172.21.15.66:0/1429830816 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7af80836d0 con 0x7f7b0015a150 2024-09-10T22:35:40.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.964+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f7b00147080 con 0x7f7b0015a150 2024-09-10T22:35:40.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.966+0000 7f7af27fc640 1 -- 172.21.15.66:0/1429830816 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f7af8088100 con 0x7f7b0015a150 2024-09-10T22:35:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.968+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ae4077f70 msgr2=0x7f7ae407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.968+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ae4077f70 0x7f7ae407a430 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7af4004510 tx=0x7f7af400b390 comp rx=0 tx=0).stop 2024-09-10T22:35:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.968+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0015a150 msgr2=0x7f7b001f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.968+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0015a150 0x7f7b001f2660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7af800ad00 tx=0x7f7af800a470 comp rx=0 tx=0).stop 2024-09-10T22:35:40.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 shutdown_connections 2024-09-10T22:35:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ae4077f70 0x7f7ae407a430 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b0015a150 0x7f7b001f2660 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b000ca6a0 0x7f7b000d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 --2- 172.21.15.66:0/1429830816 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7b0006e5f0 0x7f7b001473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 >> 172.21.15.66:0/1429830816 conn(0x7f7b000b1660 msgr2=0x7f7b000ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 shutdown_connections 2024-09-10T22:35:40.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:40.969+0000 7f7b07fd3640 1 -- 172.21.15.66:0/1429830816 wait complete. 2024-09-10T22:35:40.982 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T22:35:41.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.087+0000 7fdcca80f640 1 -- 172.21.15.66:0/283106414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcac004800 msgr2=0x7fdcac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.087+0000 7fdcca80f640 1 --2- 172.21.15.66:0/283106414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcac004800 0x7fdcac004c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8001710 tx=0x7fdcac005830 comp rx=0 tx=0).stop 2024-09-10T22:35:41.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.087+0000 7fdcca80f640 1 -- 172.21.15.66:0/283106414 shutdown_connections 2024-09-10T22:35:41.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.087+0000 7fdcca80f640 1 --2- 172.21.15.66:0/283106414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcac004800 0x7fdcac004c80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.087+0000 7fdcca80f640 1 -- 172.21.15.66:0/283106414 >> 172.21.15.66:0/283106414 conn(0x7fdcc40d3f10 msgr2=0x7fdcc40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:41.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.087+0000 7fdcca80f640 1 -- 172.21.15.66:0/283106414 shutdown_connections 2024-09-10T22:35:41.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.087+0000 7fdcca80f640 1 -- 172.21.15.66:0/283106414 wait complete. 2024-09-10T22:35:41.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.088+0000 7fdcca80f640 1 Processor -- start 2024-09-10T22:35:41.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.088+0000 7fdcca80f640 1 -- start start 2024-09-10T22:35:41.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcca80f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac004800 0x7fdcc41f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcca80f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc41f8210 0x7fdcc40c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcca80f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcc40c9b00 0x7fdcc41f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcca80f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40d1f90 con 0x7fdcc41f8210 2024-09-10T22:35:41.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcca80f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc40d4d50 con 0x7fdcac004800 2024-09-10T22:35:41.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcca80f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc40d2110 con 0x7fdcc40c9b00 2024-09-10T22:35:41.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc41f8210 0x7fdcc40c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc41f8210 0x7fdcc40c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38392/0 (socket says 172.21.15.66:38392) 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 -- 172.21.15.66:0/2932394433 learned_addr learned my addr 172.21.15.66:0/2932394433 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc3fff640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac004800 0x7fdcc41f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc8dac640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcc40c9b00 0x7fdcc41f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 -- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcc40c9b00 msgr2=0x7fdcc41f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcc40c9b00 0x7fdcc41f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 -- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac004800 msgr2=0x7fdcc41f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac004800 0x7fdcc41f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.089+0000 7fdcc37fe640 1 -- 172.21.15.66:0/2932394433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc40ca2f0 con 0x7fdcc41f8210 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.090+0000 7fdcc37fe640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc41f8210 0x7fdcc40c95c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdcb802a8e0 tx=0x7fdcb80062f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:41.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.090+0000 7fdcc0ff9640 1 -- 172.21.15.66:0/2932394433 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8003840 con 0x7fdcc41f8210 2024-09-10T22:35:41.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.090+0000 7fdcc0ff9640 1 -- 172.21.15.66:0/2932394433 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcb8039070 con 0x7fdcc41f8210 2024-09-10T22:35:41.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.090+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc41f5ab0 con 0x7fdcc41f8210 2024-09-10T22:35:41.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.090+0000 7fdcc0ff9640 1 -- 172.21.15.66:0/2932394433 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8003d70 con 0x7fdcc41f8210 2024-09-10T22:35:41.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.090+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41f6070 con 0x7fdcc41f8210 2024-09-10T22:35:41.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.091+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc4160910 con 0x7fdcc41f8210 2024-09-10T22:35:41.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.091+0000 7fdcc0ff9640 1 -- 172.21.15.66:0/2932394433 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdcb8033420 con 0x7fdcc41f8210 2024-09-10T22:35:41.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.092+0000 7fdcc0ff9640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdcb0077fa0 0x7fdcb007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.092+0000 7fdcc0ff9640 1 -- 172.21.15.66:0/2932394433 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fdcb80b9110 con 0x7fdcc41f8210 2024-09-10T22:35:41.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.092+0000 7fdcc3fff640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdcb0077fa0 0x7fdcb007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.093+0000 7fdcc3fff640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdcb0077fa0 0x7fdcb007a460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40b9a40 tx=0x7fdcb4006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:41.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.096+0000 7fdcc0ff9640 1 -- 172.21.15.66:0/2932394433 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fdcc4160910 con 0x7fdcc41f8210 2024-09-10T22:35:41.358 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.355+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fdcc414fbf0 con 0x7fdcc41f8210 2024-09-10T22:35:41.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.357+0000 7fdcc0ff9640 1 -- 172.21.15.66:0/2932394433 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7fdcc414fbf0 con 0x7fdcc41f8210 2024-09-10T22:35:41.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdcb0077fa0 msgr2=0x7fdcb007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdcb0077fa0 0x7fdcb007a460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40b9a40 tx=0x7fdcb4006040 comp rx=0 tx=0).stop 2024-09-10T22:35:41.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc41f8210 msgr2=0x7fdcc40c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc41f8210 0x7fdcc40c95c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdcb802a8e0 tx=0x7fdcb80062f0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 shutdown_connections 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fdcb0077fa0 0x7fdcb007a460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fdcc40c9b00 0x7fdcc41f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcc41f8210 0x7fdcc40c95c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 --2- 172.21.15.66:0/2932394433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdcac004800 0x7fdcc41f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 >> 172.21.15.66:0/2932394433 conn(0x7fdcc40d3f10 msgr2=0x7fdcc40b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 shutdown_connections 2024-09-10T22:35:41.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.360+0000 7fdcca80f640 1 -- 172.21.15.66:0/2932394433 wait complete. 2024-09-10T22:35:41.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:35:41.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 -- 172.21.15.66:0/943219721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb40ceda0 msgr2=0x7f7cb4072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 --2- 172.21.15.66:0/943219721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb40ceda0 0x7f7cb4072ad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005fd0 tx=0x7f7cac02aa90 comp rx=0 tx=0).stop 2024-09-10T22:35:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 -- 172.21.15.66:0/943219721 shutdown_connections 2024-09-10T22:35:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 --2- 172.21.15.66:0/943219721 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb40ceda0 0x7f7cb4072ad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 --2- 172.21.15.66:0/943219721 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cb4155270 0x7f7cb40ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 -- 172.21.15.66:0/943219721 >> 172.21.15.66:0/943219721 conn(0x7f7cb40b16d0 msgr2=0x7f7cb40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 -- 172.21.15.66:0/943219721 shutdown_connections 2024-09-10T22:35:41.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.477+0000 7f7cb8900640 1 -- 172.21.15.66:0/943219721 wait complete. 2024-09-10T22:35:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.479+0000 7f7cb8900640 1 Processor -- start 2024-09-10T22:35:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.479+0000 7f7cb8900640 1 -- start start 2024-09-10T22:35:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb8900640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cb40ceda0 0x7f7cb40c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb8900640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cb4154e90 0x7f7cb4159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb8900640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb4155270 0x7f7cb4148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb8900640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb40733e0 con 0x7f7cb4155270 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb8900640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4073260 con 0x7f7cb4154e90 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb8900640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb4073560 con 0x7f7cb40ceda0 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb4155270 0x7f7cb4148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb259c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cb40ceda0 0x7f7cb40c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb4155270 0x7f7cb4148bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38400/0 (socket says 172.21.15.66:38400) 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 -- 172.21.15.66:0/3454098255 learned_addr learned my addr 172.21.15.66:0/3454098255 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:41.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cabfff640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cb4154e90 0x7f7cb4159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 -- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cb40ceda0 msgr2=0x7f7cb40c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cb40ceda0 0x7f7cb40c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 -- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cb4154e90 msgr2=0x7f7cb4159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cb4154e90 0x7f7cb4159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 -- 172.21.15.66:0/3454098255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb40c65d0 con 0x7f7cb4155270 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.480+0000 7f7cb2d9d640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb4155270 0x7f7cb4148bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7cac006010 tx=0x7f7cac003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.481+0000 7f7caaffd640 1 -- 172.21.15.66:0/3454098255 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac046070 con 0x7f7cb4155270 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.481+0000 7f7caaffd640 1 -- 172.21.15.66:0/3454098255 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cac004040 con 0x7f7cb4155270 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.481+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb41f4d10 con 0x7f7cb4155270 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.481+0000 7f7caaffd640 1 -- 172.21.15.66:0/3454098255 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac004450 con 0x7f7cb4155270 2024-09-10T22:35:41.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.481+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb41f55b0 con 0x7f7cb4155270 2024-09-10T22:35:41.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.482+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb4073a10 con 0x7f7cb4155270 2024-09-10T22:35:41.486 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.483+0000 7f7caaffd640 1 -- 172.21.15.66:0/3454098255 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7cac0045f0 con 0x7f7cb4155270 2024-09-10T22:35:41.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.484+0000 7f7caaffd640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c94077f70 0x7f7c9407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.484+0000 7f7caaffd640 1 -- 172.21.15.66:0/3454098255 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f7cac0ba970 con 0x7f7cb4155270 2024-09-10T22:35:41.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.484+0000 7f7cb259c640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c94077f70 0x7f7c9407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.484+0000 7f7cb259c640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c94077f70 0x7f7c9407a430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40ce660 tx=0x7f7ca4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:41.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.488+0000 7f7caaffd640 1 -- 172.21.15.66:0/3454098255 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7cac032030 con 0x7f7cb4155270 2024-09-10T22:35:41.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.680+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7cb40c3000 con 0x7f7cb4155270 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.681+0000 7f7caaffd640 1 -- 172.21.15.66:0/3454098255 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f7cac083170 con 0x7f7cb4155270 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:35:41.683 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:35:41.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c94077f70 msgr2=0x7f7c9407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c94077f70 0x7f7c9407a430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40ce660 tx=0x7f7ca4009040 comp rx=0 tx=0).stop 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb4155270 msgr2=0x7f7cb4148bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb4155270 0x7f7cb4148bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7cac006010 tx=0x7f7cac003ba0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 shutdown_connections 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c94077f70 0x7f7c9407a430 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cb4155270 0x7f7cb4148bf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7cb4154e90 0x7f7cb4159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 --2- 172.21.15.66:0/3454098255 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7cb40ceda0 0x7f7cb40c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 >> 172.21.15.66:0/3454098255 conn(0x7f7cb40b16d0 msgr2=0x7f7cb40cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 shutdown_connections 2024-09-10T22:35:41.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.683+0000 7f7cb8900640 1 -- 172.21.15.66:0/3454098255 wait complete. 2024-09-10T22:35:41.697 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:35:41.697 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:35:41.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 -- 172.21.15.66:0/1293182064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d26c0 msgr2=0x7f409c162060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1293182064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d26c0 0x7f409c162060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f409c0b3690 tx=0x7f408c02a530 comp rx=0 tx=0).stop 2024-09-10T22:35:41.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 -- 172.21.15.66:0/1293182064 shutdown_connections 2024-09-10T22:35:41.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1293182064 >> v2:172.21.15.94:3300/0 conn(0x7f409c0cfc00 0x7f409c1625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1293182064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d26c0 0x7f409c162060 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 -- 172.21.15.66:0/1293182064 >> 172.21.15.66:0/1293182064 conn(0x7f409c0d3f70 msgr2=0x7f409c0d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:41.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 -- 172.21.15.66:0/1293182064 shutdown_connections 2024-09-10T22:35:41.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.807+0000 7f40a1e23640 1 -- 172.21.15.66:0/1293182064 wait complete. 2024-09-10T22:35:41.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.809+0000 7f40a1e23640 1 Processor -- start 2024-09-10T22:35:41.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.809+0000 7f40a1e23640 1 -- start start 2024-09-10T22:35:41.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f40a1e23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c0cfc00 0x7f409c14a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f40a1e23640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f409c0d26c0 0x7f409c1f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f40a1e23640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d2aa0 0x7f409c1f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f40a1e23640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f409c162ae0 con 0x7f409c0d2aa0 2024-09-10T22:35:41.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f40a1e23640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f409c0b2f70 con 0x7f409c0cfc00 2024-09-10T22:35:41.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f40a1e23640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f409c162c60 con 0x7f409c0d26c0 2024-09-10T22:35:41.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d2aa0 0x7f409c1f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d2aa0 0x7f409c1f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38422/0 (socket says 172.21.15.66:38422) 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 -- 172.21.15.66:0/1795985308 learned_addr learned my addr 172.21.15.66:0/1795985308 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409affd640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f409c0d26c0 0x7f409c1f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409b7fe640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c0cfc00 0x7f409c14a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 -- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f409c0d26c0 msgr2=0x7f409c1f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f409c0d26c0 0x7f409c1f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 -- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c0cfc00 msgr2=0x7f409c14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c0cfc00 0x7f409c14a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 -- 172.21.15.66:0/1795985308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c15a2c0 con 0x7f409c0d2aa0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f409bfff640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d2aa0 0x7f409c1f8190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f409400cb30 tx=0x7f409400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.810+0000 7f4093fff640 1 -- 172.21.15.66:0/1795985308 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094016020 con 0x7f409c0d2aa0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.811+0000 7f4093fff640 1 -- 172.21.15.66:0/1795985308 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f409400ad90 con 0x7f409c0d2aa0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.811+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c15a5a0 con 0x7f409c0d2aa0 2024-09-10T22:35:41.814 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.811+0000 7f4093fff640 1 -- 172.21.15.66:0/1795985308 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094012680 con 0x7f409c0d2aa0 2024-09-10T22:35:41.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.811+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c0b74c0 con 0x7f409c0d2aa0 2024-09-10T22:35:41.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.812+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c162060 con 0x7f409c0d2aa0 2024-09-10T22:35:41.815 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.812+0000 7f4093fff640 1 -- 172.21.15.66:0/1795985308 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4094012820 con 0x7f409c0d2aa0 2024-09-10T22:35:41.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.813+0000 7f4093fff640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f407c077f70 0x7f407c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:41.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.813+0000 7f4093fff640 1 -- 172.21.15.66:0/1795985308 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f409409a3c0 con 0x7f409c0d2aa0 2024-09-10T22:35:41.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.813+0000 7f409b7fe640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f407c077f70 0x7f407c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:41.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.813+0000 7f409b7fe640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f407c077f70 0x7f407c07a430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:41.819 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:41.817+0000 7f4093fff640 1 -- 172.21.15.66:0/1795985308 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4094020020 con 0x7f409c0d2aa0 2024-09-10T22:35:42.011 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.009+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f409c0b7690 con 0x7f409c0d2aa0 2024-09-10T22:35:42.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.632+0000 7f4093fff640 1 -- 172.21.15.66:0/1795985308 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f4094063220 con 0x7f409c0d2aa0 2024-09-10T22:35:42.634 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:35:42.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.636+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f407c077f70 msgr2=0x7f407c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:42.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.636+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f407c077f70 0x7f407c07a430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c004730 comp rx=0 tx=0).stop 2024-09-10T22:35:42.638 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.636+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d2aa0 msgr2=0x7f409c1f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.636+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d2aa0 0x7f409c1f8190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f409400cb30 tx=0x7f409400a060 comp rx=0 tx=0).stop 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.636+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 shutdown_connections 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.636+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f407c077f70 0x7f407c07a430 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f408c034000 tx=0x7f408c004730 comp rx=0 tx=0).stop 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.636+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f409c0d2aa0 0x7f409c1f8190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.637+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f409c0d26c0 0x7f409c1f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.637+0000 7f40a1e23640 1 --2- 172.21.15.66:0/1795985308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f409c0cfc00 0x7f409c14a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.637+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 >> 172.21.15.66:0/1795985308 conn(0x7f409c0d3f70 msgr2=0x7f409c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:42.639 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.637+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 shutdown_connections 2024-09-10T22:35:42.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.638+0000 7f40a1e23640 1 -- 172.21.15.66:0/1795985308 wait complete. 2024-09-10T22:35:42.654 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:35:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 -- 172.21.15.66:0/735937794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1806cb20 msgr2=0x7fee1815ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/735937794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1806cb20 0x7fee1815ffe0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee180ce6f0 tx=0x7fee1002aae0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 -- 172.21.15.66:0/735937794 shutdown_connections 2024-09-10T22:35:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/735937794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1806cb20 0x7fee1815ffe0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/735937794 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee1815bd30 0x7fee180cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/735937794 >> v2:172.21.15.94:3300/0 conn(0x7fee180cca40 0x7fee18073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 -- 172.21.15.66:0/735937794 >> 172.21.15.66:0/735937794 conn(0x7fee180b27c0 msgr2=0x7fee180b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 -- 172.21.15.66:0/735937794 shutdown_connections 2024-09-10T22:35:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.767+0000 7fee1d3ec640 1 -- 172.21.15.66:0/735937794 wait complete. 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.768+0000 7fee1d3ec640 1 Processor -- start 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.768+0000 7fee1d3ec640 1 -- start start 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee1d3ec640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee1806cb20 0x7fee181f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee1d3ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee180cca40 0x7fee180c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee1d3ec640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1815bd30 0x7fee18150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee1d3ec640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fee1806f810 con 0x7fee1815bd30 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee1d3ec640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fee1806f690 con 0x7fee180cca40 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee1d3ec640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fee1806f990 con 0x7fee1806cb20 2024-09-10T22:35:42.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1815bd30 0x7fee18150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1815bd30 0x7fee18150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38430/0 (socket says 172.21.15.66:38430) 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 -- 172.21.15.66:0/419331983 learned_addr learned my addr 172.21.15.66:0/419331983 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee0ffff640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee180cca40 0x7fee180c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee16ffd640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee1806cb20 0x7fee181f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 -- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee1806cb20 msgr2=0x7fee181f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee1806cb20 0x7fee181f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 -- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee180cca40 msgr2=0x7fee180c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee180cca40 0x7fee180c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 -- 172.21.15.66:0/419331983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee180c7180 con 0x7fee1815bd30 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.769+0000 7fee177fe640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1815bd30 0x7fee18150260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee10009d90 tx=0x7fee10003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.770+0000 7fee0f7fe640 1 -- 172.21.15.66:0/419331983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10046070 con 0x7fee1815bd30 2024-09-10T22:35:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.770+0000 7fee0f7fe640 1 -- 172.21.15.66:0/419331983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee10004250 con 0x7fee1815bd30 2024-09-10T22:35:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.770+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee180c7460 con 0x7fee1815bd30 2024-09-10T22:35:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.770+0000 7fee0f7fe640 1 -- 172.21.15.66:0/419331983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100046a0 con 0x7fee1815bd30 2024-09-10T22:35:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.770+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee181f3f90 con 0x7fee1815bd30 2024-09-10T22:35:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.771+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee1806fe40 con 0x7fee1815bd30 2024-09-10T22:35:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.771+0000 7fee0f7fe640 1 -- 172.21.15.66:0/419331983 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fee10036400 con 0x7fee1815bd30 2024-09-10T22:35:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.772+0000 7fee0f7fe640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fedf8077fa0 0x7fedf807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.772+0000 7fee0f7fe640 1 -- 172.21.15.66:0/419331983 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7fee100bad20 con 0x7fee1815bd30 2024-09-10T22:35:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.772+0000 7fee16ffd640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fedf8077fa0 0x7fedf807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.773+0000 7fee16ffd640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fedf8077fa0 0x7fedf807a460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee08005940 tx=0x7fee0800a2d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:42.778 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.777+0000 7fee0f7fe640 1 -- 172.21.15.66:0/419331983 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fee10032030 con 0x7fee1815bd30 2024-09-10T22:35:42.963 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:42.961+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fee180b3d80 con 0x7fee1815bd30 2024-09-10T22:35:43.640 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.638+0000 7fee0f7fe640 1 -- 172.21.15.66:0/419331983 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7fee1008c1b0 con 0x7fee1815bd30 2024-09-10T22:35:43.640 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fedf8077fa0 msgr2=0x7fedf807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fedf8077fa0 0x7fedf807a460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee08005940 tx=0x7fee0800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1815bd30 msgr2=0x7fee18150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1815bd30 0x7fee18150260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee10009d90 tx=0x7fee10003f10 comp rx=0 tx=0).stop 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 shutdown_connections 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fedf8077fa0 0x7fedf807a460 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee08005940 tx=0x7fee0800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee1815bd30 0x7fee18150260 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fee180cca40 0x7fee180c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 --2- 172.21.15.66:0/419331983 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fee1806cb20 0x7fee181f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.646 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.643+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 >> 172.21.15.66:0/419331983 conn(0x7fee180b27c0 msgr2=0x7fee180b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:43.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.644+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 shutdown_connections 2024-09-10T22:35:43.649 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.645+0000 7fee1d3ec640 1 -- 172.21.15.66:0/419331983 wait complete. 2024-09-10T22:35:43.672 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:35:43.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.784+0000 7fc09fb3a640 1 -- 172.21.15.66:0/854971626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 msgr2=0x7fc080004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:43.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.784+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/854971626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc080004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009b40 tx=0x7fc080005830 comp rx=0 tx=0).stop 2024-09-10T22:35:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.784+0000 7fc09fb3a640 1 -- 172.21.15.66:0/854971626 shutdown_connections 2024-09-10T22:35:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.784+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/854971626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc080004c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.784+0000 7fc09fb3a640 1 -- 172.21.15.66:0/854971626 >> 172.21.15.66:0/854971626 conn(0x7fc0980d3f10 msgr2=0x7fc0980d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.784+0000 7fc09fb3a640 1 -- 172.21.15.66:0/854971626 shutdown_connections 2024-09-10T22:35:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.784+0000 7fc09fb3a640 1 -- 172.21.15.66:0/854971626 wait complete. 2024-09-10T22:35:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.785+0000 7fc09fb3a640 1 Processor -- start 2024-09-10T22:35:43.787 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.785+0000 7fc09fb3a640 1 -- start start 2024-09-10T22:35:43.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09fb3a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc0980cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09fb3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0981f4cf0 0x7fc0981f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09fb3a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0980a1390 0x7fc09815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09fb3a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0980d4e70 con 0x7fc080004800 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09fb3a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0980d4cf0 con 0x7fc0981f4cf0 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09fb3a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc0980d23e0 con 0x7fc0980a1390 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc0980cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc0980cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52274/0 (socket says 172.21.15.66:52274) 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 -- 172.21.15.66:0/2428575415 learned_addr learned my addr 172.21.15.66:0/2428575415 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d0d5640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0981f4cf0 0x7fc0981f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09e0d7640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0980a1390 0x7fc09815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 -- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0980a1390 msgr2=0x7fc09815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0980a1390 0x7fc09815fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 -- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0981f4cf0 msgr2=0x7fc0981f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0981f4cf0 0x7fc0981f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09d8d6640 1 -- 172.21.15.66:0/2428575415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0981f8240 con 0x7fc080004800 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.786+0000 7fc09e0d7640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0980a1390 0x7fc09815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.787+0000 7fc09d8d6640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc0980cdc00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc090004d90 tx=0x7fc090001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.787+0000 7fc0967fc640 1 -- 172.21.15.66:0/2428575415 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090017070 con 0x7fc080004800 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.787+0000 7fc0967fc640 1 -- 172.21.15.66:0/2428575415 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0900108c0 con 0x7fc080004800 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.787+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0981f8520 con 0x7fc080004800 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.787+0000 7fc0967fc640 1 -- 172.21.15.66:0/2428575415 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090001d40 con 0x7fc080004800 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.787+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0981f44f0 con 0x7fc080004800 2024-09-10T22:35:43.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.788+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0980d28b0 con 0x7fc080004800 2024-09-10T22:35:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.788+0000 7fc0967fc640 1 -- 172.21.15.66:0/2428575415 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc090010cd0 con 0x7fc080004800 2024-09-10T22:35:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.789+0000 7fc0967fc640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc084077fa0 0x7fc08407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.789+0000 7fc0967fc640 1 -- 172.21.15.66:0/2428575415 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7fc090099c80 con 0x7fc080004800 2024-09-10T22:35:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.789+0000 7fc09d0d5640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc084077fa0 0x7fc08407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:43.792 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.790+0000 7fc09d0d5640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc084077fa0 0x7fc08407a460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc08c035000 tx=0x7fc08c0046c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:43.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:43.794+0000 7fc0967fc640 1 -- 172.21.15.66:0/2428575415 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc090062190 con 0x7fc080004800 2024-09-10T22:35:44.016 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.014+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc0980b6590 con 0x7fc080004800 2024-09-10T22:35:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.653+0000 7fc0967fc640 1 -- 172.21.15.66:0/2428575415 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fc090066bc0 con 0x7fc080004800 2024-09-10T22:35:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T22:35:44.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.657+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc084077fa0 msgr2=0x7fc08407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:44.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.658+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc084077fa0 0x7fc08407a460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc08c035000 tx=0x7fc08c0046c0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.658+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 msgr2=0x7fc0980cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:44.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.658+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc0980cdc00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc090004d90 tx=0x7fc090001630 comp rx=0 tx=0).stop 2024-09-10T22:35:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.661+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 shutdown_connections 2024-09-10T22:35:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.661+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc084077fa0 0x7fc08407a460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.661+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0980a1390 0x7fc09815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.661+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0981f4cf0 0x7fc0981f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.661+0000 7fc09fb3a640 1 --2- 172.21.15.66:0/2428575415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc080004800 0x7fc0980cdc00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.661+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 >> 172.21.15.66:0/2428575415 conn(0x7fc0980d3f10 msgr2=0x7fc09815e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:44.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.662+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 shutdown_connections 2024-09-10T22:35:44.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.662+0000 7fc09fb3a640 1 -- 172.21.15.66:0/2428575415 wait complete. 2024-09-10T22:35:44.685 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.791+0000 7faceef94640 1 -- 172.21.15.66:0/1982114047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face80d2be0 msgr2=0x7face80d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.791+0000 7faceef94640 1 --2- 172.21.15.66:0/1982114047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face80d2be0 0x7face80d1180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7facd8005fd0 tx=0x7facd802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.791+0000 7faceef94640 1 -- 172.21.15.66:0/1982114047 shutdown_connections 2024-09-10T22:35:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.791+0000 7faceef94640 1 --2- 172.21.15.66:0/1982114047 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face80d2be0 0x7face80d1180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.791+0000 7faceef94640 1 -- 172.21.15.66:0/1982114047 >> 172.21.15.66:0/1982114047 conn(0x7face80d3f10 msgr2=0x7face80d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.792+0000 7faceef94640 1 -- 172.21.15.66:0/1982114047 shutdown_connections 2024-09-10T22:35:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.792+0000 7faceef94640 1 -- 172.21.15.66:0/1982114047 wait complete. 2024-09-10T22:35:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.793+0000 7faceef94640 1 Processor -- start 2024-09-10T22:35:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.793+0000 7faceef94640 1 -- start start 2024-09-10T22:35:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7face80d1ae0 0x7face80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face80d2be0 0x7face81f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face815e090 0x7face8152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7face80b4ef0 con 0x7face815e090 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7face80b4d70 con 0x7face80d2be0 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7face80b5070 con 0x7face80d1ae0 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face815e090 0x7face8152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face815e090 0x7face8152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52294/0 (socket says 172.21.15.66:52294) 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 -- 172.21.15.66:0/3863489928 learned_addr learned my addr 172.21.15.66:0/3863489928 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 -- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7face80d1ae0 msgr2=0x7face80cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7face7fff640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face80d2be0 0x7face81f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7facecd30640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7face80d1ae0 0x7face80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7face80d1ae0 0x7face80cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 -- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face80d2be0 msgr2=0x7face81f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face80d2be0 0x7face81f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 -- 172.21.15.66:0/3863489928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80ce120 con 0x7face815e090 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7face7fff640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face80d2be0 0x7face81f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7facecd30640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7face80d1ae0 0x7face80cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faced531640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face815e090 0x7face8152260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7face000c440 tx=0x7face000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7face57fa640 1 -- 172.21.15.66:0/3863489928 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0011790 con 0x7face815e090 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7face57fa640 1 -- 172.21.15.66:0/3863489928 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7face0017070 con 0x7face815e090 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face81f8bc0 con 0x7face815e090 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7face57fa640 1 -- 172.21.15.66:0/3863489928 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0011cc0 con 0x7face815e090 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.794+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face80cc7d0 con 0x7face815e090 2024-09-10T22:35:44.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.795+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face80b66a0 con 0x7face815e090 2024-09-10T22:35:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.796+0000 7face57fa640 1 -- 172.21.15.66:0/3863489928 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7face001c460 con 0x7face815e090 2024-09-10T22:35:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.797+0000 7face57fa640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7facc8077f70 0x7facc807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.797+0000 7face57fa640 1 -- 172.21.15.66:0/3863489928 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7face0099b70 con 0x7face815e090 2024-09-10T22:35:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.797+0000 7facecd30640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7facc8077f70 0x7facc807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:44.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.797+0000 7facecd30640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7facc8077f70 0x7facc807a430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7face80a1c00 tx=0x7facd0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:44.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.801+0000 7face57fa640 1 -- 172.21.15.66:0/3863489928 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7face81f8bc0 con 0x7face815e090 2024-09-10T22:35:44.988 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.986+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7face80b32f0 con 0x7face815e090 2024-09-10T22:35:44.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.987+0000 7face57fa640 1 -- 172.21.15.66:0/3863489928 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16718 (secure 0 0 0) 0x7face80ce120 con 0x7face815e090 2024-09-10T22:35:44.989 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:44.989 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":31,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:44.016095+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:42.014848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:42.966140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:44.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7facc8077f70 msgr2=0x7facc807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:44.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7facc8077f70 0x7facc807a430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7face80a1c00 tx=0x7facd0009040 comp rx=0 tx=0).stop 2024-09-10T22:35:44.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face815e090 msgr2=0x7face8152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face815e090 0x7face8152260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7face000c440 tx=0x7face000c900 comp rx=0 tx=0).stop 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 shutdown_connections 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7facc8077f70 0x7facc807a430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7face815e090 0x7face8152260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7face80d2be0 0x7face81f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 --2- 172.21.15.66:0/3863489928 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7face80d1ae0 0x7face80cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 >> 172.21.15.66:0/3863489928 conn(0x7face80d3f10 msgr2=0x7face80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 shutdown_connections 2024-09-10T22:35:44.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:44.989+0000 7faceef94640 1 -- 172.21.15.66:0/3863489928 wait complete. 2024-09-10T22:35:45.002 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.106+0000 7f8d26582640 1 -- 172.21.15.66:0/397054093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d08003a00 msgr2=0x7f8d08003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.106+0000 7f8d26582640 1 --2- 172.21.15.66:0/397054093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d08003a00 0x7f8d08003e20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009b50 tx=0x7f8d08005830 comp rx=0 tx=0).stop 2024-09-10T22:35:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.106+0000 7f8d24b1f640 1 -- 172.21.15.66:0/397054093 reap_dead start 2024-09-10T22:35:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.106+0000 7f8d26582640 1 -- 172.21.15.66:0/397054093 shutdown_connections 2024-09-10T22:35:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.106+0000 7f8d26582640 1 -- 172.21.15.66:0/397054093 >> 172.21.15.66:0/397054093 conn(0x7f8d200d1d20 msgr2=0x7f8d200d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.107+0000 7f8d26582640 1 -- 172.21.15.66:0/397054093 shutdown_connections 2024-09-10T22:35:45.109 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.107+0000 7f8d26582640 1 -- 172.21.15.66:0/397054093 wait complete. 2024-09-10T22:35:45.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.108+0000 7f8d26582640 1 Processor -- start 2024-09-10T22:35:45.110 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.108+0000 7f8d26582640 1 -- start start 2024-09-10T22:35:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d26582640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d08003a00 0x7f8d200b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d26582640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200c8110 0x7f8d200b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d26582640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d200b4b50 0x7f8d200b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d26582640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2015f7f0 con 0x7f8d200c8110 2024-09-10T22:35:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d26582640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2015f670 con 0x7f8d200b4b50 2024-09-10T22:35:45.112 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d26582640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8d200ce3c0 con 0x7f8d08003a00 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200c8110 0x7f8d200b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200c8110 0x7f8d200b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52320/0 (socket says 172.21.15.66:52320) 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 -- 172.21.15.66:0/2393405682 learned_addr learned my addr 172.21.15.66:0/2393405682 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d24b1f640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d200b4b50 0x7f8d200b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1ffff640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d08003a00 0x7f8d200b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 -- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d08003a00 msgr2=0x7f8d200b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d08003a00 0x7f8d200b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 -- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d200b4b50 msgr2=0x7f8d200b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d200b4b50 0x7f8d200b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 -- 172.21.15.66:0/2393405682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d200b7180 con 0x7f8d200c8110 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.109+0000 7f8d1f7fe640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200c8110 0x7f8d200b5cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d14001240 tx=0x7f8d140039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.110+0000 7f8d1cff9640 1 -- 172.21.15.66:0/2393405682 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14015070 con 0x7f8d200c8110 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.110+0000 7f8d1cff9640 1 -- 172.21.15.66:0/2393405682 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d14011880 con 0x7f8d200c8110 2024-09-10T22:35:45.113 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.110+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d200b7460 con 0x7f8d200c8110 2024-09-10T22:35:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.110+0000 7f8d1cff9640 1 -- 172.21.15.66:0/2393405682 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14011cd0 con 0x7f8d200c8110 2024-09-10T22:35:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.110+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d200c6910 con 0x7f8d200c8110 2024-09-10T22:35:45.114 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.111+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20069620 con 0x7f8d200c8110 2024-09-10T22:35:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.111+0000 7f8d1cff9640 1 -- 172.21.15.66:0/2393405682 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8d140103c0 con 0x7f8d200c8110 2024-09-10T22:35:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.112+0000 7f8d1cff9640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8d0c077fa0 0x7f8d0c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.112+0000 7f8d1cff9640 1 -- 172.21.15.66:0/2393405682 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8d1409ad60 con 0x7f8d200c8110 2024-09-10T22:35:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.112+0000 7f8d1ffff640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8d0c077fa0 0x7f8d0c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.115 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.113+0000 7f8d1ffff640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8d0c077fa0 0x7f8d0c07a460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009b50 tx=0x7f8d1002d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:45.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.116+0000 7f8d1cff9640 1 -- 172.21.15.66:0/2393405682 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8d20069620 con 0x7f8d200c8110 2024-09-10T22:35:45.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.301+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d200c6ae0 con 0x7f8d200c8110 2024-09-10T22:35:45.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.302+0000 7f8d1cff9640 1 -- 172.21.15.66:0/2393405682 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16718 (secure 0 0 0) 0x7f8d140632e0 con 0x7f8d200c8110 2024-09-10T22:35:45.304 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:45.304 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":31,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:44.016095+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:42.014848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:42.966140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8d0c077fa0 msgr2=0x7f8d0c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8d0c077fa0 0x7f8d0c07a460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009b50 tx=0x7f8d1002d040 comp rx=0 tx=0).stop 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200c8110 msgr2=0x7f8d200b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200c8110 0x7f8d200b5cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d14001240 tx=0x7f8d140039c0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 shutdown_connections 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8d0c077fa0 0x7f8d0c07a460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d200b4b50 0x7f8d200b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d200c8110 0x7f8d200b5cf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 --2- 172.21.15.66:0/2393405682 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8d08003a00 0x7f8d200b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 >> 172.21.15.66:0/2393405682 conn(0x7f8d200d1d20 msgr2=0x7f8d2015ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:45.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 shutdown_connections 2024-09-10T22:35:45.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.304+0000 7f8d26582640 1 -- 172.21.15.66:0/2393405682 wait complete. 2024-09-10T22:35:45.317 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 -- 172.21.15.66:0/2895019947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01480cca40 msgr2=0x7f0148073740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 --2- 172.21.15.66:0/2895019947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01480cca40 0x7f0148073740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0148074660 tx=0x7f014002aae0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 -- 172.21.15.66:0/2895019947 shutdown_connections 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 --2- 172.21.15.66:0/2895019947 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014806cb20 0x7f014815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 --2- 172.21.15.66:0/2895019947 >> v2:172.21.15.94:3300/0 conn(0x7f014815bd30 0x7f01480cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 --2- 172.21.15.66:0/2895019947 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f01480cca40 0x7f0148073740 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 -- 172.21.15.66:0/2895019947 >> 172.21.15.66:0/2895019947 conn(0x7f01480b27c0 msgr2=0x7f01480b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 -- 172.21.15.66:0/2895019947 shutdown_connections 2024-09-10T22:35:45.426 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.423+0000 7f014feb8640 1 -- 172.21.15.66:0/2895019947 wait complete. 2024-09-10T22:35:45.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 Processor -- start 2024-09-10T22:35:45.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 -- start start 2024-09-10T22:35:45.428 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014806cb20 0x7f014814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01480cca40 0x7f01480ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f014815bd30 0x7f0148147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f014806f810 con 0x7f014815bd30 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f014806f690 con 0x7f01480cca40 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014feb8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f014806f990 con 0x7f014806cb20 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014e455640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f014815bd30 0x7f0148147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014e455640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f014815bd30 0x7f0148147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52338/0 (socket says 172.21.15.66:52338) 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.425+0000 7f014e455640 1 -- 172.21.15.66:0/3059747576 learned_addr learned my addr 172.21.15.66:0/3059747576 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014d453640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01480cca40 0x7f01480ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014dc54640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014806cb20 0x7f014814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014e455640 1 -- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014806cb20 msgr2=0x7f014814aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014e455640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014806cb20 0x7f014814aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014e455640 1 -- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01480cca40 msgr2=0x7f01480ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:45.429 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014e455640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01480cca40 0x7f01480ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014e455640 1 -- 172.21.15.66:0/3059747576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014814afb0 con 0x7f014815bd30 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014dc54640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014806cb20 0x7f014814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014e455640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f014815bd30 0x7f0148147750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f013800d650 tx=0x7f013800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f01467fc640 1 -- 172.21.15.66:0/3059747576 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138017070 con 0x7f014815bd30 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f01467fc640 1 -- 172.21.15.66:0/3059747576 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0138005310 con 0x7f014815bd30 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01480ca640 con 0x7f014815bd30 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f01467fc640 1 -- 172.21.15.66:0/3059747576 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138009820 con 0x7f014815bd30 2024-09-10T22:35:45.430 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.426+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01480c9620 con 0x7f014815bd30 2024-09-10T22:35:45.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.427+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014806fe40 con 0x7f014815bd30 2024-09-10T22:35:45.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.428+0000 7f01467fc640 1 -- 172.21.15.66:0/3059747576 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f01380099c0 con 0x7f014815bd30 2024-09-10T22:35:45.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.428+0000 7f01467fc640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0134077fa0 0x7f013407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:45.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.428+0000 7f01467fc640 1 -- 172.21.15.66:0/3059747576 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0138099d20 con 0x7f014815bd30 2024-09-10T22:35:45.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.428+0000 7f014dc54640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0134077fa0 0x7f013407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:45.431 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.429+0000 7f014dc54640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0134077fa0 0x7f013407a460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0140035000 tx=0x7f0140039040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:45.434 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.432+0000 7f01467fc640 1 -- 172.21.15.66:0/3059747576 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f014814afb0 con 0x7f014815bd30 2024-09-10T22:35:45.658 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:45.656+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f01480c8d50 con 0x7f014815bd30 2024-09-10T22:35:46.657 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.651+0000 7f01467fc640 1 -- 172.21.15.66:0/3059747576 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f0138067e20 con 0x7f014815bd30 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0134077fa0 msgr2=0x7f013407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0134077fa0 0x7f013407a460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0140035000 tx=0x7f0140039040 comp rx=0 tx=0).stop 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f014815bd30 msgr2=0x7f0148147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f014815bd30 0x7f0148147750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f013800d650 tx=0x7f013800db10 comp rx=0 tx=0).stop 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 shutdown_connections 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0134077fa0 0x7f013407a460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f014815bd30 0x7f0148147750 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01480cca40 0x7f01480ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 --2- 172.21.15.66:0/3059747576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f014806cb20 0x7f014814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 >> 172.21.15.66:0/3059747576 conn(0x7f01480b27c0 msgr2=0x7f01480b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:46.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 shutdown_connections 2024-09-10T22:35:46.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.654+0000 7f014feb8640 1 -- 172.21.15.66:0/3059747576 wait complete. 2024-09-10T22:35:46.805 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:35:46.805 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:35:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.925+0000 7fe21a7da640 1 -- 172.21.15.66:0/1253133905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 msgr2=0x7fe21415a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.925+0000 7fe21a7da640 1 --2- 172.21.15.66:0/1253133905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 0x7fe21415a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe208005fd0 tx=0x7fe20802b8f0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.925+0000 7fe21a7da640 1 -- 172.21.15.66:0/1253133905 shutdown_connections 2024-09-10T22:35:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.925+0000 7fe21a7da640 1 --2- 172.21.15.66:0/1253133905 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 0x7fe21415a840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.925+0000 7fe21a7da640 1 --2- 172.21.15.66:0/1253133905 >> v2:172.21.15.94:3300/0 conn(0x7fe2141512b0 0x7fe2140ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.925+0000 7fe21a7da640 1 -- 172.21.15.66:0/1253133905 >> 172.21.15.66:0/1253133905 conn(0x7fe2140b16d0 msgr2=0x7fe2140b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:46.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.926+0000 7fe21a7da640 1 -- 172.21.15.66:0/1253133905 shutdown_connections 2024-09-10T22:35:46.929 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.926+0000 7fe21a7da640 1 -- 172.21.15.66:0/1253133905 wait complete. 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.927+0000 7fe21a7da640 1 Processor -- start 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.927+0000 7fe21a7da640 1 -- start start 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe21a7da640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 0x7fe2140c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe21a7da640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2141512b0 0x7fe2141f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe21a7da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21415ad80 0x7fe21414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe21a7da640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe2140cc5a0 con 0x7fe2141512b0 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe21a7da640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe21415b230 con 0x7fe21415ad80 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe21a7da640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe2140cc720 con 0x7fe2140ca6b0 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe2137fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2141512b0 0x7fe2141f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe213fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 0x7fe2140c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:46.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe213fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 0x7fe2140c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3301/0 says I am v2:172.21.15.66:49634/0 (socket says 172.21.15.66:49634) 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe213fff640 1 -- 172.21.15.66:0/2610550678 learned_addr learned my addr 172.21.15.66:0/2610550678 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe218d77640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21415ad80 0x7fe21414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe2137fe640 1 -- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 msgr2=0x7fe2140c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe2137fe640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 0x7fe2140c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe2137fe640 1 -- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21415ad80 msgr2=0x7fe21414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe2137fe640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21415ad80 0x7fe21414d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe2137fe640 1 -- 172.21.15.66:0/2610550678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141f6f80 con 0x7fe2141512b0 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.928+0000 7fe2137fe640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2141512b0 0x7fe2141f6a40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe208033000 tx=0x7fe208003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.929+0000 7fe210ff9640 1 -- 172.21.15.66:0/2610550678 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208040030 con 0x7fe2141512b0 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.929+0000 7fe210ff9640 1 -- 172.21.15.66:0/2610550678 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe208003b80 con 0x7fe2141512b0 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.929+0000 7fe210ff9640 1 -- 172.21.15.66:0/2610550678 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208003dd0 con 0x7fe2141512b0 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.929+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2140c4550 con 0x7fe2141512b0 2024-09-10T22:35:46.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.929+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2140c4b10 con 0x7fe2141512b0 2024-09-10T22:35:46.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.930+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21415a840 con 0x7fe2141512b0 2024-09-10T22:35:46.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.930+0000 7fe210ff9640 1 -- 172.21.15.66:0/2610550678 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe20800b040 con 0x7fe2141512b0 2024-09-10T22:35:46.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.931+0000 7fe210ff9640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe200077f70 0x7fe20007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:46.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.931+0000 7fe210ff9640 1 -- 172.21.15.66:0/2610550678 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe2080bab00 con 0x7fe2141512b0 2024-09-10T22:35:46.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.931+0000 7fe213fff640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe200077f70 0x7fe20007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:46.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.932+0000 7fe213fff640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe200077f70 0x7fe20007a430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2040095f0 tx=0x7fe204005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:46.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:46.935+0000 7fe210ff9640 1 -- 172.21.15.66:0/2610550678 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe2141f6f80 con 0x7fe2141512b0 2024-09-10T22:35:47.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.157+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe2141592a0 con 0x7fe2141512b0 2024-09-10T22:35:47.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.657+0000 7fe210ff9640 1 -- 172.21.15.66:0/2610550678 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fe208082f90 con 0x7fe2141512b0 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe200077f70 msgr2=0x7fe20007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe200077f70 0x7fe20007a430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2040095f0 tx=0x7fe204005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2141512b0 msgr2=0x7fe2141f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2141512b0 0x7fe2141f6a40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe208033000 tx=0x7fe208003670 comp rx=0 tx=0).stop 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 shutdown_connections 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe200077f70 0x7fe20007a430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe21415ad80 0x7fe21414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2141512b0 0x7fe2141f6a40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 --2- 172.21.15.66:0/2610550678 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe2140ca6b0 0x7fe2140c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 >> 172.21.15.66:0/2610550678 conn(0x7fe2140b16d0 msgr2=0x7fe2140b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:47.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 shutdown_connections 2024-09-10T22:35:47.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.660+0000 7fe21a7da640 1 -- 172.21.15.66:0/2610550678 wait complete. 2024-09-10T22:35:47.675 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:35:47.675 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:47.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 -- 172.21.15.66:0/3605530450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78680cca30 msgr2=0x7f78680afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:47.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 --2- 172.21.15.66:0/3605530450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78680cca30 0x7f78680afac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7850009830 tx=0x7f785002a470 comp rx=0 tx=0).stop 2024-09-10T22:35:47.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 -- 172.21.15.66:0/3605530450 shutdown_connections 2024-09-10T22:35:47.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 --2- 172.21.15.66:0/3605530450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f78680cca30 0x7f78680afac0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 --2- 172.21.15.66:0/3605530450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78680748e0 0x7f78680b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 -- 172.21.15.66:0/3605530450 >> 172.21.15.66:0/3605530450 conn(0x7f78680d0430 msgr2=0x7f78680d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:47.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 -- 172.21.15.66:0/3605530450 shutdown_connections 2024-09-10T22:35:47.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.783+0000 7f786ccfe640 1 -- 172.21.15.66:0/3605530450 wait complete. 2024-09-10T22:35:47.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.784+0000 7f786ccfe640 1 Processor -- start 2024-09-10T22:35:47.786 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.784+0000 7f786ccfe640 1 -- start start 2024-09-10T22:35:47.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f786ccfe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78680748e0 0x7f7868155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:47.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f786ccfe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78680cca30 0x7f78681549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:47.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f786ccfe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786815e4c0 0x7f78681f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:47.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f786ccfe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f78680af780 con 0x7f786815e4c0 2024-09-10T22:35:47.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f786ccfe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f78680af900 con 0x7f78680748e0 2024-09-10T22:35:47.788 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f786ccfe640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f78680c93d0 con 0x7f78680cca30 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786815e4c0 0x7f78681f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786815e4c0 0x7f78681f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52372/0 (socket says 172.21.15.66:52372) 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 -- 172.21.15.66:0/192064867 learned_addr learned my addr 172.21.15.66:0/192064867 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f785ffff640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78680cca30 0x7f78681549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f786659c640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78680748e0 0x7f7868155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 -- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78680cca30 msgr2=0x7f78681549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78680cca30 0x7f78681549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 -- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78680748e0 msgr2=0x7f7868155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78680748e0 0x7f7868155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.785+0000 7f7866d9d640 1 -- 172.21.15.66:0/192064867 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78680b5d80 con 0x7f786815e4c0 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.786+0000 7f7866d9d640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786815e4c0 0x7f78681f5800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f786000cb00 tx=0x7f786000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.786+0000 7f785effd640 1 -- 172.21.15.66:0/192064867 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860003990 con 0x7f786815e4c0 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.786+0000 7f785effd640 1 -- 172.21.15.66:0/192064867 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7860003c40 con 0x7f786815e4c0 2024-09-10T22:35:47.789 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.786+0000 7f785effd640 1 -- 172.21.15.66:0/192064867 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860004290 con 0x7f786815e4c0 2024-09-10T22:35:47.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.786+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78680b6060 con 0x7f786815e4c0 2024-09-10T22:35:47.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.786+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7868155690 con 0x7f786815e4c0 2024-09-10T22:35:47.790 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.787+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786806a070 con 0x7f786815e4c0 2024-09-10T22:35:47.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.787+0000 7f785effd640 1 -- 172.21.15.66:0/192064867 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7860004430 con 0x7f786815e4c0 2024-09-10T22:35:47.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.788+0000 7f785effd640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7848077f70 0x7f784807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:47.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.788+0000 7f785effd640 1 -- 172.21.15.66:0/192064867 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f786009a460 con 0x7f786815e4c0 2024-09-10T22:35:47.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.788+0000 7f786659c640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7848077f70 0x7f784807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:47.791 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.789+0000 7f786659c640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7848077f70 0x7f784807a430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7858000f60 tx=0x7f7858009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:47.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:47.792+0000 7f785effd640 1 -- 172.21.15.66:0/192064867 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f786806a070 con 0x7f786815e4c0 2024-09-10T22:35:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.016+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78680b08e0 con 0x7f786815e4c0 2024-09-10T22:35:48.018 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.016+0000 7f785effd640 1 -- 172.21.15.66:0/192064867 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f7860014070 con 0x7f786815e4c0 2024-09-10T22:35:48.018 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:48.018 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"btime":"2024-09-10T22:35:47:657322+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/307839984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":307839984},{"type":"v1","addr":"172.21.15.94:6836","nonce":307839984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2042084416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2042084416},{"type":"v1","addr":"172.21.15.94:6837","nonce":2042084416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1521655589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1521655589},{"type":"v1","addr":"172.21.15.66:6834","nonce":1521655589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:47.159794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4846},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4846":{"gid":4846,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1970715211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1970715211},{"type":"v1","addr":"172.21.15.66:6835","nonce":1970715211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4846,"qdb_cluster":[4846]},"id":2}]} 2024-09-10T22:35:48.019 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7848077f70 msgr2=0x7f784807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7848077f70 0x7f784807a430 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7858000f60 tx=0x7f7858009040 comp rx=0 tx=0).stop 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786815e4c0 msgr2=0x7f78681f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786815e4c0 0x7f78681f5800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f786000cb00 tx=0x7f786000a250 comp rx=0 tx=0).stop 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 shutdown_connections 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7848077f70 0x7f784807a430 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f786815e4c0 0x7f78681f5800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f78680cca30 0x7f78681549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 --2- 172.21.15.66:0/192064867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78680748e0 0x7f7868155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 >> 172.21.15.66:0/192064867 conn(0x7f78680d0430 msgr2=0x7f786806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:48.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 shutdown_connections 2024-09-10T22:35:48.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.018+0000 7f786ccfe640 1 -- 172.21.15.66:0/192064867 wait complete. 2024-09-10T22:35:48.032 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:35:48.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 -- 172.21.15.66:0/4263265294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c0cdc00 msgr2=0x7fe03c0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 --2- 172.21.15.66:0/4263265294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c0cdc00 0x7fe03c0ccc80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009900 tx=0x7fe02c02a4d0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 -- 172.21.15.66:0/4263265294 shutdown_connections 2024-09-10T22:35:48.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 --2- 172.21.15.66:0/4263265294 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe03c0ce6a0 0x7fe03c0ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 --2- 172.21.15.66:0/4263265294 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c0cdc00 0x7fe03c0ccc80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 -- 172.21.15.66:0/4263265294 >> 172.21.15.66:0/4263265294 conn(0x7fe03c0d1d20 msgr2=0x7fe03c0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:48.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 -- 172.21.15.66:0/4263265294 shutdown_connections 2024-09-10T22:35:48.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.135+0000 7fe043ebd640 1 -- 172.21.15.66:0/4263265294 wait complete. 2024-09-10T22:35:48.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.136+0000 7fe043ebd640 1 Processor -- start 2024-09-10T22:35:48.138 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.136+0000 7fe043ebd640 1 -- start start 2024-09-10T22:35:48.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe043ebd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03c0cdc00 0x7fe03c0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe043ebd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe03c0ce6a0 0x7fe03c1f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe043ebd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c161640 0x7fe03c1f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe043ebd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c06daf0 con 0x7fe03c161640 2024-09-10T22:35:48.140 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe043ebd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c06d970 con 0x7fe03c0cdc00 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe043ebd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe03c06dc70 con 0x7fe03c0ce6a0 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c161640 0x7fe03c1f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c161640 0x7fe03c1f5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52386/0 (socket says 172.21.15.66:52386) 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 -- 172.21.15.66:0/2165756997 learned_addr learned my addr 172.21.15.66:0/2165756997 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe041458640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe03c0ce6a0 0x7fe03c1f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe041c59640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03c0cdc00 0x7fe03c0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 -- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe03c0ce6a0 msgr2=0x7fe03c1f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe03c0ce6a0 0x7fe03c1f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 -- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03c0cdc00 msgr2=0x7fe03c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03c0cdc00 0x7fe03c0b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 -- 172.21.15.66:0/2165756997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c0b6630 con 0x7fe03c161640 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.137+0000 7fe04245a640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c161640 0x7fe03c1f5100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe03400cb00 tx=0x7fe03400a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:48.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.138+0000 7fe0327fc640 1 -- 172.21.15.66:0/2165756997 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034004b20 con 0x7fe03c161640 2024-09-10T22:35:48.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.138+0000 7fe0327fc640 1 -- 172.21.15.66:0/2165756997 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe034004db0 con 0x7fe03c161640 2024-09-10T22:35:48.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.138+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c0b6910 con 0x7fe03c161640 2024-09-10T22:35:48.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.138+0000 7fe0327fc640 1 -- 172.21.15.66:0/2165756997 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034005280 con 0x7fe03c161640 2024-09-10T22:35:48.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.138+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c0cafb0 con 0x7fe03c161640 2024-09-10T22:35:48.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.139+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03c06e200 con 0x7fe03c161640 2024-09-10T22:35:48.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.139+0000 7fe0327fc640 1 -- 172.21.15.66:0/2165756997 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe034005420 con 0x7fe03c161640 2024-09-10T22:35:48.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.140+0000 7fe0327fc640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe020077f70 0x7fe02007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.140+0000 7fe0327fc640 1 -- 172.21.15.66:0/2165756997 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe03409a580 con 0x7fe03c161640 2024-09-10T22:35:48.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.140+0000 7fe041c59640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe020077f70 0x7fe02007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.141+0000 7fe041c59640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe020077f70 0x7fe02007a430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe038006fd0 tx=0x7fe038008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:48.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.144+0000 7fe0327fc640 1 -- 172.21.15.66:0/2165756997 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe034014030 con 0x7fe03c161640 2024-09-10T22:35:48.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.330+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe03c0cf6b0 con 0x7fe03c161640 2024-09-10T22:35:48.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.331+0000 7fe0327fc640 1 -- 172.21.15.66:0/2165756997 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16705 (secure 0 0 0) 0x7fe034063080 con 0x7fe03c161640 2024-09-10T22:35:48.334 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:48.334 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":32,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:45.646689+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:42.014848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:42.966140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe020077f70 msgr2=0x7fe02007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe020077f70 0x7fe02007a430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe038006fd0 tx=0x7fe038008040 comp rx=0 tx=0).stop 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c161640 msgr2=0x7fe03c1f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c161640 0x7fe03c1f5100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe03400cb00 tx=0x7fe03400a250 comp rx=0 tx=0).stop 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 shutdown_connections 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe020077f70 0x7fe02007a430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03c161640 0x7fe03c1f5100 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe03c0ce6a0 0x7fe03c1f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 --2- 172.21.15.66:0/2165756997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe03c0cdc00 0x7fe03c0b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.333+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 >> 172.21.15.66:0/2165756997 conn(0x7fe03c0d1d20 msgr2=0x7fe03c0757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:48.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.334+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 shutdown_connections 2024-09-10T22:35:48.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.334+0000 7fe043ebd640 1 -- 172.21.15.66:0/2165756997 wait complete. 2024-09-10T22:35:48.347 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:35:48.347 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:48.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/862311109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b0160350 msgr2=0x7ff7b00cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/862311109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b0160350 0x7ff7b00cbc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8005fd0 tx=0x7ff7a802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/862311109 shutdown_connections 2024-09-10T22:35:48.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/862311109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b0160350 0x7ff7b00cbc70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/862311109 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7b00b1650 0x7ff7b00688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.453 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/862311109 >> 172.21.15.66:0/862311109 conn(0x7ff7b00d0430 msgr2=0x7ff7b00d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:48.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/862311109 shutdown_connections 2024-09-10T22:35:48.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.450+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/862311109 wait complete. 2024-09-10T22:35:48.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.451+0000 7ff7b4f4b640 1 Processor -- start 2024-09-10T22:35:48.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.451+0000 7ff7b4f4b640 1 -- start start 2024-09-10T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7b4f4b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7b00b1650 0x7ff7b01f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7b4f4b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b015be30 0x7ff7b00b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7b4f4b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b0160350 0x7ff7b00cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7b4f4b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b00cc330 con 0x7ff7b015be30 2024-09-10T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7b4f4b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b00cc1b0 con 0x7ff7b0160350 2024-09-10T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7b4f4b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b00cc4b0 con 0x7ff7b00b1650 2024-09-10T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b015be30 0x7ff7b00b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b015be30 0x7ff7b00b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52408/0 (socket says 172.21.15.66:52408) 2024-09-10T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 -- 172.21.15.66:0/3372139469 learned_addr learned my addr 172.21.15.66:0/3372139469 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7ae59c640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7b00b1650 0x7ff7b01f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7aed9d640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b0160350 0x7ff7b00cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 -- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7b00b1650 msgr2=0x7ff7b01f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7b00b1650 0x7ff7b01f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 -- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b0160350 msgr2=0x7ff7b00cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b0160350 0x7ff7b00cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.452+0000 7ff7a7fff640 1 -- 172.21.15.66:0/3372139469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b00ccdd0 con 0x7ff7b015be30 2024-09-10T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.453+0000 7ff7a7fff640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b015be30 0x7ff7b00b3a00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c8c0 tx=0x7ff79800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.453+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/3372139469 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798007bb0 con 0x7ff7b015be30 2024-09-10T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.453+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/3372139469 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff798007e40 con 0x7ff7b015be30 2024-09-10T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.453+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b00ce0a0 con 0x7ff7b015be30 2024-09-10T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.453+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/3372139469 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79800f690 con 0x7ff7b015be30 2024-09-10T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.453+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b00ce660 con 0x7ff7b015be30 2024-09-10T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.454+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b00688e0 con 0x7ff7b015be30 2024-09-10T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.454+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/3372139469 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff798020030 con 0x7ff7b015be30 2024-09-10T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.455+0000 7ff7a6ffd640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff790077f70 0x7ff79007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.455+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/3372139469 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff79809a0a0 con 0x7ff7b015be30 2024-09-10T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.455+0000 7ff7ae59c640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff790077f70 0x7ff79007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.455+0000 7ff7ae59c640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff790077f70 0x7ff79007a430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0007fd0 tx=0x7ff7a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.459+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/3372139469 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff798014030 con 0x7ff7b015be30 2024-09-10T22:35:48.643 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.641+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7b006d810 con 0x7ff790077f70 2024-09-10T22:35:48.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.646+0000 7ff7a6ffd640 1 -- 172.21.15.66:0/3372139469 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7ff7b006d810 con 0x7ff790077f70 2024-09-10T22:35:48.651 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:48.652 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff790077f70 msgr2=0x7ff79007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff790077f70 0x7ff79007a430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0007fd0 tx=0x7ff7a0009040 comp rx=0 tx=0).stop 2024-09-10T22:35:48.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b015be30 msgr2=0x7ff7b00b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b015be30 0x7ff7b00b3a00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c8c0 tx=0x7ff79800cd80 comp rx=0 tx=0).stop 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 shutdown_connections 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff790077f70 0x7ff79007a430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b0160350 0x7ff7b00cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff7b015be30 0x7ff7b00b3a00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 --2- 172.21.15.66:0/3372139469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff7b00b1650 0x7ff7b01f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 >> 172.21.15.66:0/3372139469 conn(0x7ff7b00d0430 msgr2=0x7ff7b01582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 shutdown_connections 2024-09-10T22:35:48.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.649+0000 7ff7b4f4b640 1 -- 172.21.15.66:0/3372139469 wait complete. 2024-09-10T22:35:48.662 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-10T22:35:48.602668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240988,"kb_used_data":3916,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733732,"statfs":{"total":773094113280,"available":772847341568,"internally_reserved":0,"allocated":4009984,"data_stored":2238160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.967962"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661269+0000","last_change":"2024-09-10T22:35:44.658461+0000","last_active":"2024-09-10T22:35:44.661269+0000","last_peered":"2024-09-10T22:35:44.661269+0000","last_clean":"2024-09-10T22:35:44.661269+0000","last_became_active":"2024-09-10T22:35:44.658183+0000","last_became_peered":"2024-09-10T22:35:44.658183+0000","last_unstale":"2024-09-10T22:35:44.661269+0000","last_undegraded":"2024-09-10T22:35:44.661269+0000","last_fullsized":"2024-09-10T22:35:44.661269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:52.718822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670511+0000","last_change":"2024-09-10T22:35:44.661094+0000","last_active":"2024-09-10T22:35:44.670511+0000","last_peered":"2024-09-10T22:35:44.670511+0000","last_clean":"2024-09-10T22:35:44.670511+0000","last_became_active":"2024-09-10T22:35:44.659854+0000","last_became_peered":"2024-09-10T22:35:44.659854+0000","last_unstale":"2024-09-10T22:35:44.670511+0000","last_undegraded":"2024-09-10T22:35:44.670511+0000","last_fullsized":"2024-09-10T22:35:44.670511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:01.410053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654686+0000","last_change":"2024-09-10T22:35:44.655880+0000","last_active":"2024-09-10T22:35:45.654686+0000","last_peered":"2024-09-10T22:35:45.654686+0000","last_clean":"2024-09-10T22:35:45.654686+0000","last_became_active":"2024-09-10T22:35:44.655601+0000","last_became_peered":"2024-09-10T22:35:44.655601+0000","last_unstale":"2024-09-10T22:35:45.654686+0000","last_undegraded":"2024-09-10T22:35:45.654686+0000","last_fullsized":"2024-09-10T22:35:45.654686+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:33.161604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669617+0000","last_change":"2024-09-10T22:35:44.664831+0000","last_active":"2024-09-10T22:35:44.669617+0000","last_peered":"2024-09-10T22:35:44.669617+0000","last_clean":"2024-09-10T22:35:44.669617+0000","last_became_active":"2024-09-10T22:35:44.664238+0000","last_became_peered":"2024-09-10T22:35:44.664238+0000","last_unstale":"2024-09-10T22:35:44.669617+0000","last_undegraded":"2024-09-10T22:35:44.669617+0000","last_fullsized":"2024-09-10T22:35:44.669617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:14.187410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670178+0000","last_change":"2024-09-10T22:35:44.666874+0000","last_active":"2024-09-10T22:35:44.670178+0000","last_peered":"2024-09-10T22:35:44.670178+0000","last_clean":"2024-09-10T22:35:44.670178+0000","last_became_active":"2024-09-10T22:35:44.666541+0000","last_became_peered":"2024-09-10T22:35:44.666541+0000","last_unstale":"2024-09-10T22:35:44.670178+0000","last_undegraded":"2024-09-10T22:35:44.670178+0000","last_fullsized":"2024-09-10T22:35:44.670178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:00.279279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670660+0000","last_change":"2024-09-10T22:35:44.663837+0000","last_active":"2024-09-10T22:35:44.670660+0000","last_peered":"2024-09-10T22:35:44.670660+0000","last_clean":"2024-09-10T22:35:44.670660+0000","last_became_active":"2024-09-10T22:35:44.660799+0000","last_became_peered":"2024-09-10T22:35:44.660799+0000","last_unstale":"2024-09-10T22:35:44.670660+0000","last_undegraded":"2024-09-10T22:35:44.670660+0000","last_fullsized":"2024-09-10T22:35:44.670660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:33.385136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661763+0000","last_change":"2024-09-10T22:35:44.659433+0000","last_active":"2024-09-10T22:35:44.661763+0000","last_peered":"2024-09-10T22:35:44.661763+0000","last_clean":"2024-09-10T22:35:44.661763+0000","last_became_active":"2024-09-10T22:35:44.659079+0000","last_became_peered":"2024-09-10T22:35:44.659079+0000","last_unstale":"2024-09-10T22:35:44.661763+0000","last_undegraded":"2024-09-10T22:35:44.661763+0000","last_fullsized":"2024-09-10T22:35:44.661763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:13.884564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666521+0000","last_change":"2024-09-10T22:35:44.662626+0000","last_active":"2024-09-10T22:35:44.666521+0000","last_peered":"2024-09-10T22:35:44.666521+0000","last_clean":"2024-09-10T22:35:44.666521+0000","last_became_active":"2024-09-10T22:35:44.662241+0000","last_became_peered":"2024-09-10T22:35:44.662241+0000","last_unstale":"2024-09-10T22:35:44.666521+0000","last_undegraded":"2024-09-10T22:35:44.666521+0000","last_fullsized":"2024-09-10T22:35:44.666521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:48.022021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655343+0000","last_change":"2024-09-10T22:35:44.657732+0000","last_active":"2024-09-10T22:35:45.655343+0000","last_peered":"2024-09-10T22:35:45.655343+0000","last_clean":"2024-09-10T22:35:45.655343+0000","last_became_active":"2024-09-10T22:35:44.655449+0000","last_became_peered":"2024-09-10T22:35:44.655449+0000","last_unstale":"2024-09-10T22:35:45.655343+0000","last_undegraded":"2024-09-10T22:35:45.655343+0000","last_fullsized":"2024-09-10T22:35:45.655343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:42.465443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.672409+0000","last_change":"2024-09-10T22:35:44.669432+0000","last_active":"2024-09-10T22:35:44.672409+0000","last_peered":"2024-09-10T22:35:44.672409+0000","last_clean":"2024-09-10T22:35:44.672409+0000","last_became_active":"2024-09-10T22:35:44.668907+0000","last_became_peered":"2024-09-10T22:35:44.668907+0000","last_unstale":"2024-09-10T22:35:44.672409+0000","last_undegraded":"2024-09-10T22:35:44.672409+0000","last_fullsized":"2024-09-10T22:35:44.672409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:50.077818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.660845+0000","last_change":"2024-09-10T22:35:44.658442+0000","last_active":"2024-09-10T22:35:44.660845+0000","last_peered":"2024-09-10T22:35:44.660845+0000","last_clean":"2024-09-10T22:35:44.660845+0000","last_became_active":"2024-09-10T22:35:44.658163+0000","last_became_peered":"2024-09-10T22:35:44.658163+0000","last_unstale":"2024-09-10T22:35:44.660845+0000","last_undegraded":"2024-09-10T22:35:44.660845+0000","last_fullsized":"2024-09-10T22:35:44.660845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:29.247536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654317+0000","last_change":"2024-09-10T22:35:44.665674+0000","last_active":"2024-09-10T22:35:45.654317+0000","last_peered":"2024-09-10T22:35:45.654317+0000","last_clean":"2024-09-10T22:35:45.654317+0000","last_became_active":"2024-09-10T22:35:44.665440+0000","last_became_peered":"2024-09-10T22:35:44.665440+0000","last_unstale":"2024-09-10T22:35:45.654317+0000","last_undegraded":"2024-09-10T22:35:45.654317+0000","last_fullsized":"2024-09-10T22:35:45.654317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:36.866074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670152+0000","last_change":"2024-09-10T22:35:44.665750+0000","last_active":"2024-09-10T22:35:44.670152+0000","last_peered":"2024-09-10T22:35:44.670152+0000","last_clean":"2024-09-10T22:35:44.670152+0000","last_became_active":"2024-09-10T22:35:44.665475+0000","last_became_peered":"2024-09-10T22:35:44.665475+0000","last_unstale":"2024-09-10T22:35:44.670152+0000","last_undegraded":"2024-09-10T22:35:44.670152+0000","last_fullsized":"2024-09-10T22:35:44.670152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:12.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670634+0000","last_change":"2024-09-10T22:35:44.660754+0000","last_active":"2024-09-10T22:35:44.670634+0000","last_peered":"2024-09-10T22:35:44.670634+0000","last_clean":"2024-09-10T22:35:44.670634+0000","last_became_active":"2024-09-10T22:35:44.660453+0000","last_became_peered":"2024-09-10T22:35:44.660453+0000","last_unstale":"2024-09-10T22:35:44.670634+0000","last_undegraded":"2024-09-10T22:35:44.670634+0000","last_fullsized":"2024-09-10T22:35:44.670634+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:22.577095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653915+0000","last_change":"2024-09-10T22:35:44.654084+0000","last_active":"2024-09-10T22:35:45.653915+0000","last_peered":"2024-09-10T22:35:45.653915+0000","last_clean":"2024-09-10T22:35:45.653915+0000","last_became_active":"2024-09-10T22:35:44.653681+0000","last_became_peered":"2024-09-10T22:35:44.653681+0000","last_unstale":"2024-09-10T22:35:45.653915+0000","last_undegraded":"2024-09-10T22:35:45.653915+0000","last_fullsized":"2024-09-10T22:35:45.653915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:25.907466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670003+0000","last_change":"2024-09-10T22:35:44.668702+0000","last_active":"2024-09-10T22:35:44.670003+0000","last_peered":"2024-09-10T22:35:44.670003+0000","last_clean":"2024-09-10T22:35:44.670003+0000","last_became_active":"2024-09-10T22:35:44.668405+0000","last_became_peered":"2024-09-10T22:35:44.668405+0000","last_unstale":"2024-09-10T22:35:44.670003+0000","last_undegraded":"2024-09-10T22:35:44.670003+0000","last_fullsized":"2024-09-10T22:35:44.670003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:42.465443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666460+0000","last_change":"2024-09-10T22:35:44.662295+0000","last_active":"2024-09-10T22:35:44.666460+0000","last_peered":"2024-09-10T22:35:44.666460+0000","last_clean":"2024-09-10T22:35:44.666460+0000","last_became_active":"2024-09-10T22:35:44.661967+0000","last_became_peered":"2024-09-10T22:35:44.661967+0000","last_unstale":"2024-09-10T22:35:44.666460+0000","last_undegraded":"2024-09-10T22:35:44.666460+0000","last_fullsized":"2024-09-10T22:35:44.666460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:14.941677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669489+0000","last_change":"2024-09-10T22:35:44.660421+0000","last_active":"2024-09-10T22:35:44.669489+0000","last_peered":"2024-09-10T22:35:44.669489+0000","last_clean":"2024-09-10T22:35:44.669489+0000","last_became_active":"2024-09-10T22:35:44.659490+0000","last_became_peered":"2024-09-10T22:35:44.659490+0000","last_unstale":"2024-09-10T22:35:44.669489+0000","last_undegraded":"2024-09-10T22:35:44.669489+0000","last_fullsized":"2024-09-10T22:35:44.669489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:50.476281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.675054+0000","last_change":"2024-09-10T22:35:44.672017+0000","last_active":"2024-09-10T22:35:44.675054+0000","last_peered":"2024-09-10T22:35:44.675054+0000","last_clean":"2024-09-10T22:35:44.675054+0000","last_became_active":"2024-09-10T22:35:44.671729+0000","last_became_peered":"2024-09-10T22:35:44.671729+0000","last_unstale":"2024-09-10T22:35:44.675054+0000","last_undegraded":"2024-09-10T22:35:44.675054+0000","last_fullsized":"2024-09-10T22:35:44.675054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:55.472544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.665189+0000","last_change":"2024-09-10T22:35:44.662346+0000","last_active":"2024-09-10T22:35:44.665189+0000","last_peered":"2024-09-10T22:35:44.665189+0000","last_clean":"2024-09-10T22:35:44.665189+0000","last_became_active":"2024-09-10T22:35:44.662064+0000","last_became_peered":"2024-09-10T22:35:44.662064+0000","last_unstale":"2024-09-10T22:35:44.665189+0000","last_undegraded":"2024-09-10T22:35:44.665189+0000","last_fullsized":"2024-09-10T22:35:44.665189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:25.361336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655299+0000","last_change":"2024-09-10T22:35:44.656870+0000","last_active":"2024-09-10T22:35:45.655299+0000","last_peered":"2024-09-10T22:35:45.655299+0000","last_clean":"2024-09-10T22:35:45.655299+0000","last_became_active":"2024-09-10T22:35:44.656109+0000","last_became_peered":"2024-09-10T22:35:44.656109+0000","last_unstale":"2024-09-10T22:35:45.655299+0000","last_undegraded":"2024-09-10T22:35:45.655299+0000","last_fullsized":"2024-09-10T22:35:45.655299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:01.275798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654873+0000","last_change":"2024-09-10T22:35:44.666261+0000","last_active":"2024-09-10T22:35:45.654873+0000","last_peered":"2024-09-10T22:35:45.654873+0000","last_clean":"2024-09-10T22:35:45.654873+0000","last_became_active":"2024-09-10T22:35:44.665965+0000","last_became_peered":"2024-09-10T22:35:44.665965+0000","last_unstale":"2024-09-10T22:35:45.654873+0000","last_undegraded":"2024-09-10T22:35:45.654873+0000","last_fullsized":"2024-09-10T22:35:45.654873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:15.559224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652660+0000","last_change":"2024-09-10T22:35:44.655485+0000","last_active":"2024-09-10T22:35:45.652660+0000","last_peered":"2024-09-10T22:35:45.652660+0000","last_clean":"2024-09-10T22:35:45.652660+0000","last_became_active":"2024-09-10T22:35:44.655176+0000","last_became_peered":"2024-09-10T22:35:44.655176+0000","last_unstale":"2024-09-10T22:35:45.652660+0000","last_undegraded":"2024-09-10T22:35:45.652660+0000","last_fullsized":"2024-09-10T22:35:45.652660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:02.543958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653681+0000","last_change":"2024-09-10T22:35:44.665023+0000","last_active":"2024-09-10T22:35:45.653681+0000","last_peered":"2024-09-10T22:35:45.653681+0000","last_clean":"2024-09-10T22:35:45.653681+0000","last_became_active":"2024-09-10T22:35:44.664755+0000","last_became_peered":"2024-09-10T22:35:44.664755+0000","last_unstale":"2024-09-10T22:35:45.653681+0000","last_undegraded":"2024-09-10T22:35:45.653681+0000","last_fullsized":"2024-09-10T22:35:45.653681+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:11.295729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669684+0000","last_change":"2024-09-10T22:35:44.658308+0000","last_active":"2024-09-10T22:35:44.669684+0000","last_peered":"2024-09-10T22:35:44.669684+0000","last_clean":"2024-09-10T22:35:44.669684+0000","last_became_active":"2024-09-10T22:35:44.657751+0000","last_became_peered":"2024-09-10T22:35:44.657751+0000","last_unstale":"2024-09-10T22:35:44.669684+0000","last_undegraded":"2024-09-10T22:35:44.669684+0000","last_fullsized":"2024-09-10T22:35:44.669684+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:46.030439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669462+0000","last_change":"2024-09-10T22:35:44.660112+0000","last_active":"2024-09-10T22:35:44.669462+0000","last_peered":"2024-09-10T22:35:44.669462+0000","last_clean":"2024-09-10T22:35:44.669462+0000","last_became_active":"2024-09-10T22:35:44.659148+0000","last_became_peered":"2024-09-10T22:35:44.659148+0000","last_unstale":"2024-09-10T22:35:44.669462+0000","last_undegraded":"2024-09-10T22:35:44.669462+0000","last_fullsized":"2024-09-10T22:35:44.669462+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:54.559529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669628+0000","last_change":"2024-09-10T22:35:44.664434+0000","last_active":"2024-09-10T22:35:44.669628+0000","last_peered":"2024-09-10T22:35:44.669628+0000","last_clean":"2024-09-10T22:35:44.669628+0000","last_became_active":"2024-09-10T22:35:44.663532+0000","last_became_peered":"2024-09-10T22:35:44.663532+0000","last_unstale":"2024-09-10T22:35:44.669628+0000","last_undegraded":"2024-09-10T22:35:44.669628+0000","last_fullsized":"2024-09-10T22:35:44.669628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:51.866302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654570+0000","last_change":"2024-09-10T22:35:44.652979+0000","last_active":"2024-09-10T22:35:45.654570+0000","last_peered":"2024-09-10T22:35:45.654570+0000","last_clean":"2024-09-10T22:35:45.654570+0000","last_became_active":"2024-09-10T22:35:44.652607+0000","last_became_peered":"2024-09-10T22:35:44.652607+0000","last_unstale":"2024-09-10T22:35:45.654570+0000","last_undegraded":"2024-09-10T22:35:45.654570+0000","last_fullsized":"2024-09-10T22:35:45.654570+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.032728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655317+0000","last_change":"2024-09-10T22:35:44.666266+0000","last_active":"2024-09-10T22:35:45.655317+0000","last_peered":"2024-09-10T22:35:45.655317+0000","last_clean":"2024-09-10T22:35:45.655317+0000","last_became_active":"2024-09-10T22:35:44.665974+0000","last_became_peered":"2024-09-10T22:35:44.665974+0000","last_unstale":"2024-09-10T22:35:45.655317+0000","last_undegraded":"2024-09-10T22:35:45.655317+0000","last_fullsized":"2024-09-10T22:35:45.655317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:20.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661698+0000","last_change":"2024-09-10T22:35:44.659157+0000","last_active":"2024-09-10T22:35:44.661698+0000","last_peered":"2024-09-10T22:35:44.661698+0000","last_clean":"2024-09-10T22:35:44.661698+0000","last_became_active":"2024-09-10T22:35:44.658863+0000","last_became_peered":"2024-09-10T22:35:44.658863+0000","last_unstale":"2024-09-10T22:35:44.661698+0000","last_undegraded":"2024-09-10T22:35:44.661698+0000","last_fullsized":"2024-09-10T22:35:44.661698+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:59.418237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.662298+0000","last_change":"2024-09-10T22:35:44.660953+0000","last_active":"2024-09-10T22:35:44.662298+0000","last_peered":"2024-09-10T22:35:44.662298+0000","last_clean":"2024-09-10T22:35:44.662298+0000","last_became_active":"2024-09-10T22:35:44.660694+0000","last_became_peered":"2024-09-10T22:35:44.660694+0000","last_unstale":"2024-09-10T22:35:44.662298+0000","last_undegraded":"2024-09-10T22:35:44.662298+0000","last_fullsized":"2024-09-10T22:35:44.662298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:59.131318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666499+0000","last_change":"2024-09-10T22:35:44.662495+0000","last_active":"2024-09-10T22:35:44.666499+0000","last_peered":"2024-09-10T22:35:44.666499+0000","last_clean":"2024-09-10T22:35:44.666499+0000","last_became_active":"2024-09-10T22:35:44.661887+0000","last_became_peered":"2024-09-10T22:35:44.661887+0000","last_unstale":"2024-09-10T22:35:44.666499+0000","last_undegraded":"2024-09-10T22:35:44.666499+0000","last_fullsized":"2024-09-10T22:35:44.666499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:19.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652879+0000","last_change":"2024-09-10T22:35:44.656320+0000","last_active":"2024-09-10T22:35:45.652879+0000","last_peered":"2024-09-10T22:35:45.652879+0000","last_clean":"2024-09-10T22:35:45.652879+0000","last_became_active":"2024-09-10T22:35:44.655990+0000","last_became_peered":"2024-09-10T22:35:44.655990+0000","last_unstale":"2024-09-10T22:35:45.652879+0000","last_undegraded":"2024-09-10T22:35:45.652879+0000","last_fullsized":"2024-09-10T22:35:45.652879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:12.936488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670070+0000","last_change":"2024-09-10T22:35:43.650422+0000","last_active":"2024-09-10T22:35:44.670070+0000","last_peered":"2024-09-10T22:35:44.670070+0000","last_clean":"2024-09-10T22:35:44.670070+0000","last_became_active":"2024-09-10T22:35:43.650163+0000","last_became_peered":"2024-09-10T22:35:43.650163+0000","last_unstale":"2024-09-10T22:35:44.670070+0000","last_undegraded":"2024-09-10T22:35:44.670070+0000","last_fullsized":"2024-09-10T22:35:44.670070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:46.829650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661291+0000","last_change":"2024-09-10T22:35:44.659187+0000","last_active":"2024-09-10T22:35:44.661291+0000","last_peered":"2024-09-10T22:35:44.661291+0000","last_clean":"2024-09-10T22:35:44.661291+0000","last_became_active":"2024-09-10T22:35:44.658902+0000","last_became_peered":"2024-09-10T22:35:44.658902+0000","last_unstale":"2024-09-10T22:35:44.661291+0000","last_undegraded":"2024-09-10T22:35:44.661291+0000","last_fullsized":"2024-09-10T22:35:44.661291+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.095500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652805+0000","last_change":"2024-09-10T22:35:43.656995+0000","last_active":"2024-09-10T22:35:45.652805+0000","last_peered":"2024-09-10T22:35:45.652805+0000","last_clean":"2024-09-10T22:35:45.652805+0000","last_became_active":"2024-09-10T22:35:43.656678+0000","last_became_peered":"2024-09-10T22:35:43.656678+0000","last_unstale":"2024-09-10T22:35:45.652805+0000","last_undegraded":"2024-09-10T22:35:45.652805+0000","last_fullsized":"2024-09-10T22:35:45.652805+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:01.441828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655357+0000","last_change":"2024-09-10T22:35:44.664813+0000","last_active":"2024-09-10T22:35:45.655357+0000","last_peered":"2024-09-10T22:35:45.655357+0000","last_clean":"2024-09-10T22:35:45.655357+0000","last_became_active":"2024-09-10T22:35:44.664482+0000","last_became_peered":"2024-09-10T22:35:44.664482+0000","last_unstale":"2024-09-10T22:35:45.655357+0000","last_undegraded":"2024-09-10T22:35:45.655357+0000","last_fullsized":"2024-09-10T22:35:45.655357+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:48.022021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.673898+0000","last_change":"2024-09-10T22:35:43.663727+0000","last_active":"2024-09-10T22:35:44.673898+0000","last_peered":"2024-09-10T22:35:44.673898+0000","last_clean":"2024-09-10T22:35:44.673898+0000","last_became_active":"2024-09-10T22:35:43.663418+0000","last_became_peered":"2024-09-10T22:35:43.663418+0000","last_unstale":"2024-09-10T22:35:44.673898+0000","last_undegraded":"2024-09-10T22:35:44.673898+0000","last_fullsized":"2024-09-10T22:35:44.673898+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:21.593943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669601+0000","last_change":"2024-09-10T22:35:44.658719+0000","last_active":"2024-09-10T22:35:44.669601+0000","last_peered":"2024-09-10T22:35:44.669601+0000","last_clean":"2024-09-10T22:35:44.669601+0000","last_became_active":"2024-09-10T22:35:44.658397+0000","last_became_peered":"2024-09-10T22:35:44.658397+0000","last_unstale":"2024-09-10T22:35:44.669601+0000","last_undegraded":"2024-09-10T22:35:44.669601+0000","last_fullsized":"2024-09-10T22:35:44.669601+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:49:53.725746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.671442+0000","last_change":"2024-09-10T22:35:43.656397+0000","last_active":"2024-09-10T22:35:44.671442+0000","last_peered":"2024-09-10T22:35:44.671442+0000","last_clean":"2024-09-10T22:35:44.671442+0000","last_became_active":"2024-09-10T22:35:43.656125+0000","last_became_peered":"2024-09-10T22:35:43.656125+0000","last_unstale":"2024-09-10T22:35:44.671442+0000","last_undegraded":"2024-09-10T22:35:44.671442+0000","last_fullsized":"2024-09-10T22:35:44.671442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:55.949105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663373+0000","last_change":"2024-09-10T22:35:44.659872+0000","last_active":"2024-09-10T22:35:44.663373+0000","last_peered":"2024-09-10T22:35:44.663373+0000","last_clean":"2024-09-10T22:35:44.663373+0000","last_became_active":"2024-09-10T22:35:44.659602+0000","last_became_peered":"2024-09-10T22:35:44.659602+0000","last_unstale":"2024-09-10T22:35:44.663373+0000","last_undegraded":"2024-09-10T22:35:44.663373+0000","last_fullsized":"2024-09-10T22:35:44.663373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:20.838308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655433+0000","last_change":"2024-09-10T22:35:43.659132+0000","last_active":"2024-09-10T22:35:45.655433+0000","last_peered":"2024-09-10T22:35:45.655433+0000","last_clean":"2024-09-10T22:35:45.655433+0000","last_became_active":"2024-09-10T22:35:43.658844+0000","last_became_peered":"2024-09-10T22:35:43.658844+0000","last_unstale":"2024-09-10T22:35:45.655433+0000","last_undegraded":"2024-09-10T22:35:45.655433+0000","last_fullsized":"2024-09-10T22:35:45.655433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:02.734433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654653+0000","last_change":"2024-09-10T22:35:44.648187+0000","last_active":"2024-09-10T22:35:45.654653+0000","last_peered":"2024-09-10T22:35:45.654653+0000","last_clean":"2024-09-10T22:35:45.654653+0000","last_became_active":"2024-09-10T22:35:44.647909+0000","last_became_peered":"2024-09-10T22:35:44.647909+0000","last_unstale":"2024-09-10T22:35:45.654653+0000","last_undegraded":"2024-09-10T22:35:45.654653+0000","last_fullsized":"2024-09-10T22:35:45.654653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:15.818568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655023+0000","last_change":"2024-09-10T22:35:43.657036+0000","last_active":"2024-09-10T22:35:45.655023+0000","last_peered":"2024-09-10T22:35:45.655023+0000","last_clean":"2024-09-10T22:35:45.655023+0000","last_became_active":"2024-09-10T22:35:43.656794+0000","last_became_peered":"2024-09-10T22:35:43.656794+0000","last_unstale":"2024-09-10T22:35:45.655023+0000","last_undegraded":"2024-09-10T22:35:45.655023+0000","last_fullsized":"2024-09-10T22:35:45.655023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:02.923542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653981+0000","last_change":"2024-09-10T22:35:44.655450+0000","last_active":"2024-09-10T22:35:45.653981+0000","last_peered":"2024-09-10T22:35:45.653981+0000","last_clean":"2024-09-10T22:35:45.653981+0000","last_became_active":"2024-09-10T22:35:44.655177+0000","last_became_peered":"2024-09-10T22:35:44.655177+0000","last_unstale":"2024-09-10T22:35:45.653981+0000","last_undegraded":"2024-09-10T22:35:45.653981+0000","last_fullsized":"2024-09-10T22:35:45.653981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:12.936488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.649035+0000","last_change":"2024-09-10T22:35:43.659764+0000","last_active":"2024-09-10T22:35:44.649035+0000","last_peered":"2024-09-10T22:35:44.649035+0000","last_clean":"2024-09-10T22:35:44.649035+0000","last_became_active":"2024-09-10T22:35:43.659427+0000","last_became_peered":"2024-09-10T22:35:43.659427+0000","last_unstale":"2024-09-10T22:35:44.649035+0000","last_undegraded":"2024-09-10T22:35:44.649035+0000","last_fullsized":"2024-09-10T22:35:44.649035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:58.208840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670504+0000","last_change":"2024-09-10T22:35:44.663062+0000","last_active":"2024-09-10T22:35:44.670504+0000","last_peered":"2024-09-10T22:35:44.670504+0000","last_clean":"2024-09-10T22:35:44.670504+0000","last_became_active":"2024-09-10T22:35:44.662797+0000","last_became_peered":"2024-09-10T22:35:44.662797+0000","last_unstale":"2024-09-10T22:35:44.670504+0000","last_undegraded":"2024-09-10T22:35:44.670504+0000","last_fullsized":"2024-09-10T22:35:44.670504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:46.161284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653970+0000","last_change":"2024-09-10T22:35:43.649850+0000","last_active":"2024-09-10T22:35:45.653970+0000","last_peered":"2024-09-10T22:35:45.653970+0000","last_clean":"2024-09-10T22:35:45.653970+0000","last_became_active":"2024-09-10T22:35:43.649556+0000","last_became_peered":"2024-09-10T22:35:43.649556+0000","last_unstale":"2024-09-10T22:35:45.653970+0000","last_undegraded":"2024-09-10T22:35:45.653970+0000","last_fullsized":"2024-09-10T22:35:45.653970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:53.970339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654921+0000","last_change":"2024-09-10T22:35:44.671682+0000","last_active":"2024-09-10T22:35:45.654921+0000","last_peered":"2024-09-10T22:35:45.654921+0000","last_clean":"2024-09-10T22:35:45.654921+0000","last_became_active":"2024-09-10T22:35:44.671387+0000","last_became_peered":"2024-09-10T22:35:44.671387+0000","last_unstale":"2024-09-10T22:35:45.654921+0000","last_undegraded":"2024-09-10T22:35:45.654921+0000","last_fullsized":"2024-09-10T22:35:45.654921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:28.996396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.654590+0000","last_change":"2024-09-10T22:35:43.661676+0000","last_active":"2024-09-10T22:35:44.654590+0000","last_peered":"2024-09-10T22:35:44.654590+0000","last_clean":"2024-09-10T22:35:44.654590+0000","last_became_active":"2024-09-10T22:35:43.661191+0000","last_became_peered":"2024-09-10T22:35:43.661191+0000","last_unstale":"2024-09-10T22:35:44.654590+0000","last_undegraded":"2024-09-10T22:35:44.654590+0000","last_fullsized":"2024-09-10T22:35:44.654590+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:39.996112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.675010+0000","last_change":"2024-09-10T22:35:44.672583+0000","last_active":"2024-09-10T22:35:44.675010+0000","last_peered":"2024-09-10T22:35:44.675010+0000","last_clean":"2024-09-10T22:35:44.675010+0000","last_became_active":"2024-09-10T22:35:44.672310+0000","last_became_peered":"2024-09-10T22:35:44.672310+0000","last_unstale":"2024-09-10T22:35:44.675010+0000","last_undegraded":"2024-09-10T22:35:44.675010+0000","last_fullsized":"2024-09-10T22:35:44.675010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:13.748432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652906+0000","last_change":"2024-09-10T22:35:43.661493+0000","last_active":"2024-09-10T22:35:45.652906+0000","last_peered":"2024-09-10T22:35:45.652906+0000","last_clean":"2024-09-10T22:35:45.652906+0000","last_became_active":"2024-09-10T22:35:43.661224+0000","last_became_peered":"2024-09-10T22:35:43.661224+0000","last_unstale":"2024-09-10T22:35:45.652906+0000","last_undegraded":"2024-09-10T22:35:45.652906+0000","last_fullsized":"2024-09-10T22:35:45.652906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:19.090906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670417+0000","last_change":"2024-09-10T22:35:44.661923+0000","last_active":"2024-09-10T22:35:44.670417+0000","last_peered":"2024-09-10T22:35:44.670417+0000","last_clean":"2024-09-10T22:35:44.670417+0000","last_became_active":"2024-09-10T22:35:44.661622+0000","last_became_peered":"2024-09-10T22:35:44.661622+0000","last_unstale":"2024-09-10T22:35:44.670417+0000","last_undegraded":"2024-09-10T22:35:44.670417+0000","last_fullsized":"2024-09-10T22:35:44.670417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:57.051422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654830+0000","last_change":"2024-09-10T22:35:43.657210+0000","last_active":"2024-09-10T22:35:45.654830+0000","last_peered":"2024-09-10T22:35:45.654830+0000","last_clean":"2024-09-10T22:35:45.654830+0000","last_became_active":"2024-09-10T22:35:43.656950+0000","last_became_peered":"2024-09-10T22:35:43.656950+0000","last_unstale":"2024-09-10T22:35:45.654830+0000","last_undegraded":"2024-09-10T22:35:45.654830+0000","last_fullsized":"2024-09-10T22:35:45.654830+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:12.879969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654998+0000","last_change":"2024-09-10T22:35:44.671824+0000","last_active":"2024-09-10T22:35:45.654998+0000","last_peered":"2024-09-10T22:35:45.654998+0000","last_clean":"2024-09-10T22:35:45.654998+0000","last_became_active":"2024-09-10T22:35:44.671590+0000","last_became_peered":"2024-09-10T22:35:44.671590+0000","last_unstale":"2024-09-10T22:35:45.654998+0000","last_undegraded":"2024-09-10T22:35:45.654998+0000","last_fullsized":"2024-09-10T22:35:45.654998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:46.220293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.676456+0000","last_change":"2024-09-10T22:35:43.663926+0000","last_active":"2024-09-10T22:35:44.676456+0000","last_peered":"2024-09-10T22:35:44.676456+0000","last_clean":"2024-09-10T22:35:44.676456+0000","last_became_active":"2024-09-10T22:35:43.663615+0000","last_became_peered":"2024-09-10T22:35:43.663615+0000","last_unstale":"2024-09-10T22:35:44.676456+0000","last_undegraded":"2024-09-10T22:35:44.676456+0000","last_fullsized":"2024-09-10T22:35:44.676456+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:12.306113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670634+0000","last_change":"2024-09-10T22:35:44.666504+0000","last_active":"2024-09-10T22:35:44.670634+0000","last_peered":"2024-09-10T22:35:44.670634+0000","last_clean":"2024-09-10T22:35:44.670634+0000","last_became_active":"2024-09-10T22:35:44.666149+0000","last_became_peered":"2024-09-10T22:35:44.666149+0000","last_unstale":"2024-09-10T22:35:44.670634+0000","last_undegraded":"2024-09-10T22:35:44.670634+0000","last_fullsized":"2024-09-10T22:35:44.670634+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:09.758004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655557+0000","last_change":"2024-09-10T22:35:43.654467+0000","last_active":"2024-09-10T22:35:45.655557+0000","last_peered":"2024-09-10T22:35:45.655557+0000","last_clean":"2024-09-10T22:35:45.655557+0000","last_became_active":"2024-09-10T22:35:43.653474+0000","last_became_peered":"2024-09-10T22:35:43.653474+0000","last_unstale":"2024-09-10T22:35:45.655557+0000","last_undegraded":"2024-09-10T22:35:45.655557+0000","last_fullsized":"2024-09-10T22:35:45.655557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:47.327430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670757+0000","last_change":"2024-09-10T22:35:44.668271+0000","last_active":"2024-09-10T22:35:44.670757+0000","last_peered":"2024-09-10T22:35:44.670757+0000","last_clean":"2024-09-10T22:35:44.670757+0000","last_became_active":"2024-09-10T22:35:44.668001+0000","last_became_peered":"2024-09-10T22:35:44.668001+0000","last_unstale":"2024-09-10T22:35:44.670757+0000","last_undegraded":"2024-09-10T22:35:44.670757+0000","last_fullsized":"2024-09-10T22:35:44.670757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:38.801166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651424+0000","last_change":"2024-09-10T22:35:43.657001+0000","last_active":"2024-09-10T22:35:44.651424+0000","last_peered":"2024-09-10T22:35:44.651424+0000","last_clean":"2024-09-10T22:35:44.651424+0000","last_became_active":"2024-09-10T22:35:43.656698+0000","last_became_peered":"2024-09-10T22:35:43.656698+0000","last_unstale":"2024-09-10T22:35:44.651424+0000","last_undegraded":"2024-09-10T22:35:44.651424+0000","last_fullsized":"2024-09-10T22:35:44.651424+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:28.894829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666540+0000","last_change":"2024-09-10T22:35:44.662824+0000","last_active":"2024-09-10T22:35:44.666540+0000","last_peered":"2024-09-10T22:35:44.666540+0000","last_clean":"2024-09-10T22:35:44.666540+0000","last_became_active":"2024-09-10T22:35:44.661179+0000","last_became_peered":"2024-09-10T22:35:44.661179+0000","last_unstale":"2024-09-10T22:35:44.666540+0000","last_undegraded":"2024-09-10T22:35:44.666540+0000","last_fullsized":"2024-09-10T22:35:44.666540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:20.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.649246+0000","last_change":"2024-09-10T22:35:43.661113+0000","last_active":"2024-09-10T22:35:44.649246+0000","last_peered":"2024-09-10T22:35:44.649246+0000","last_clean":"2024-09-10T22:35:44.649246+0000","last_became_active":"2024-09-10T22:35:43.660808+0000","last_became_peered":"2024-09-10T22:35:43.660808+0000","last_unstale":"2024-09-10T22:35:44.649246+0000","last_undegraded":"2024-09-10T22:35:44.649246+0000","last_fullsized":"2024-09-10T22:35:44.649246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:26.752833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661362+0000","last_change":"2024-09-10T22:35:44.660082+0000","last_active":"2024-09-10T22:35:44.661362+0000","last_peered":"2024-09-10T22:35:44.661362+0000","last_clean":"2024-09-10T22:35:44.661362+0000","last_became_active":"2024-09-10T22:35:44.659694+0000","last_became_peered":"2024-09-10T22:35:44.659694+0000","last_unstale":"2024-09-10T22:35:44.661362+0000","last_undegraded":"2024-09-10T22:35:44.661362+0000","last_fullsized":"2024-09-10T22:35:44.661362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:23.444938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.674657+0000","last_change":"2024-09-10T22:35:43.663130+0000","last_active":"2024-09-10T22:35:44.674657+0000","last_peered":"2024-09-10T22:35:44.674657+0000","last_clean":"2024-09-10T22:35:44.674657+0000","last_became_active":"2024-09-10T22:35:43.662833+0000","last_became_peered":"2024-09-10T22:35:43.662833+0000","last_unstale":"2024-09-10T22:35:44.674657+0000","last_undegraded":"2024-09-10T22:35:44.674657+0000","last_fullsized":"2024-09-10T22:35:44.674657+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:01.801641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655003+0000","last_change":"2024-09-10T22:35:44.654315+0000","last_active":"2024-09-10T22:35:45.655003+0000","last_peered":"2024-09-10T22:35:45.655003+0000","last_clean":"2024-09-10T22:35:45.655003+0000","last_became_active":"2024-09-10T22:35:44.654040+0000","last_became_peered":"2024-09-10T22:35:44.654040+0000","last_unstale":"2024-09-10T22:35:45.655003+0000","last_undegraded":"2024-09-10T22:35:45.655003+0000","last_fullsized":"2024-09-10T22:35:45.655003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:43.324344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.674597+0000","last_change":"2024-09-10T22:35:43.661835+0000","last_active":"2024-09-10T22:35:44.674597+0000","last_peered":"2024-09-10T22:35:44.674597+0000","last_clean":"2024-09-10T22:35:44.674597+0000","last_became_active":"2024-09-10T22:35:43.661517+0000","last_became_peered":"2024-09-10T22:35:43.661517+0000","last_unstale":"2024-09-10T22:35:44.674597+0000","last_undegraded":"2024-09-10T22:35:44.674597+0000","last_fullsized":"2024-09-10T22:35:44.674597+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:14.571477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654590+0000","last_change":"2024-09-10T22:35:44.661067+0000","last_active":"2024-09-10T22:35:45.654590+0000","last_peered":"2024-09-10T22:35:45.654590+0000","last_clean":"2024-09-10T22:35:45.654590+0000","last_became_active":"2024-09-10T22:35:44.660755+0000","last_became_peered":"2024-09-10T22:35:44.660755+0000","last_unstale":"2024-09-10T22:35:45.654590+0000","last_undegraded":"2024-09-10T22:35:45.654590+0000","last_fullsized":"2024-09-10T22:35:45.654590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:10.408250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655131+0000","last_change":"2024-09-10T22:35:43.650404+0000","last_active":"2024-09-10T22:35:45.655131+0000","last_peered":"2024-09-10T22:35:45.655131+0000","last_clean":"2024-09-10T22:35:45.655131+0000","last_became_active":"2024-09-10T22:35:43.650111+0000","last_became_peered":"2024-09-10T22:35:43.650111+0000","last_unstale":"2024-09-10T22:35:45.655131+0000","last_undegraded":"2024-09-10T22:35:45.655131+0000","last_fullsized":"2024-09-10T22:35:45.655131+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:37.492342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669590+0000","last_change":"2024-09-10T22:35:44.664534+0000","last_active":"2024-09-10T22:35:44.669590+0000","last_peered":"2024-09-10T22:35:44.669590+0000","last_clean":"2024-09-10T22:35:44.669590+0000","last_became_active":"2024-09-10T22:35:44.663897+0000","last_became_peered":"2024-09-10T22:35:44.663897+0000","last_unstale":"2024-09-10T22:35:44.669590+0000","last_undegraded":"2024-09-10T22:35:44.669590+0000","last_fullsized":"2024-09-10T22:35:44.669590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:57.268790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655565+0000","last_change":"2024-09-10T22:35:43.650715+0000","last_active":"2024-09-10T22:35:45.655565+0000","last_peered":"2024-09-10T22:35:45.655565+0000","last_clean":"2024-09-10T22:35:45.655565+0000","last_became_active":"2024-09-10T22:35:43.650412+0000","last_became_peered":"2024-09-10T22:35:43.650412+0000","last_unstale":"2024-09-10T22:35:45.655565+0000","last_undegraded":"2024-09-10T22:35:45.655565+0000","last_fullsized":"2024-09-10T22:35:45.655565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:05.509208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663287+0000","last_change":"2024-09-10T22:35:44.660874+0000","last_active":"2024-09-10T22:35:44.663287+0000","last_peered":"2024-09-10T22:35:44.663287+0000","last_clean":"2024-09-10T22:35:44.663287+0000","last_became_active":"2024-09-10T22:35:44.659700+0000","last_became_peered":"2024-09-10T22:35:44.659700+0000","last_unstale":"2024-09-10T22:35:44.663287+0000","last_undegraded":"2024-09-10T22:35:44.663287+0000","last_fullsized":"2024-09-10T22:35:44.663287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:45.580791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655301+0000","last_change":"2024-09-10T22:35:43.659284+0000","last_active":"2024-09-10T22:35:45.655301+0000","last_peered":"2024-09-10T22:35:45.655301+0000","last_clean":"2024-09-10T22:35:45.655301+0000","last_became_active":"2024-09-10T22:35:43.659001+0000","last_became_peered":"2024-09-10T22:35:43.659001+0000","last_unstale":"2024-09-10T22:35:45.655301+0000","last_undegraded":"2024-09-10T22:35:45.655301+0000","last_fullsized":"2024-09-10T22:35:45.655301+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:51.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663311+0000","last_change":"2024-09-10T22:35:44.660625+0000","last_active":"2024-09-10T22:35:44.663311+0000","last_peered":"2024-09-10T22:35:44.663311+0000","last_clean":"2024-09-10T22:35:44.663311+0000","last_became_active":"2024-09-10T22:35:44.660325+0000","last_became_peered":"2024-09-10T22:35:44.660325+0000","last_unstale":"2024-09-10T22:35:44.663311+0000","last_undegraded":"2024-09-10T22:35:44.663311+0000","last_fullsized":"2024-09-10T22:35:44.663311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:19.135367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652962+0000","last_change":"2024-09-10T22:35:43.659599+0000","last_active":"2024-09-10T22:35:45.652962+0000","last_peered":"2024-09-10T22:35:45.652962+0000","last_clean":"2024-09-10T22:35:45.652962+0000","last_became_active":"2024-09-10T22:35:43.659258+0000","last_became_peered":"2024-09-10T22:35:43.659258+0000","last_unstale":"2024-09-10T22:35:45.652962+0000","last_undegraded":"2024-09-10T22:35:45.652962+0000","last_fullsized":"2024-09-10T22:35:45.652962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:47.327430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663244+0000","last_change":"2024-09-10T22:35:44.659463+0000","last_active":"2024-09-10T22:35:44.663244+0000","last_peered":"2024-09-10T22:35:44.663244+0000","last_clean":"2024-09-10T22:35:44.663244+0000","last_became_active":"2024-09-10T22:35:44.659151+0000","last_became_peered":"2024-09-10T22:35:44.659151+0000","last_unstale":"2024-09-10T22:35:44.663244+0000","last_undegraded":"2024-09-10T22:35:44.663244+0000","last_fullsized":"2024-09-10T22:35:44.663244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:02.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651565+0000","last_change":"2024-09-10T22:35:43.659671+0000","last_active":"2024-09-10T22:35:44.651565+0000","last_peered":"2024-09-10T22:35:44.651565+0000","last_clean":"2024-09-10T22:35:44.651565+0000","last_became_active":"2024-09-10T22:35:43.659273+0000","last_became_peered":"2024-09-10T22:35:43.659273+0000","last_unstale":"2024-09-10T22:35:44.651565+0000","last_undegraded":"2024-09-10T22:35:44.651565+0000","last_fullsized":"2024-09-10T22:35:44.651565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:47.327430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663661+0000","last_change":"2024-09-10T22:35:44.660247+0000","last_active":"2024-09-10T22:35:44.663661+0000","last_peered":"2024-09-10T22:35:44.663661+0000","last_clean":"2024-09-10T22:35:44.663661+0000","last_became_active":"2024-09-10T22:35:44.659974+0000","last_became_peered":"2024-09-10T22:35:44.659974+0000","last_unstale":"2024-09-10T22:35:44.663661+0000","last_undegraded":"2024-09-10T22:35:44.663661+0000","last_fullsized":"2024-09-10T22:35:44.663661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:08.855764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654137+0000","last_change":"2024-09-10T22:35:43.656257+0000","last_active":"2024-09-10T22:35:45.654137+0000","last_peered":"2024-09-10T22:35:45.654137+0000","last_clean":"2024-09-10T22:35:45.654137+0000","last_became_active":"2024-09-10T22:35:43.655985+0000","last_became_peered":"2024-09-10T22:35:43.655985+0000","last_unstale":"2024-09-10T22:35:45.654137+0000","last_undegraded":"2024-09-10T22:35:45.654137+0000","last_fullsized":"2024-09-10T22:35:45.654137+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:46.829650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661786+0000","last_change":"2024-09-10T22:35:44.659257+0000","last_active":"2024-09-10T22:35:44.661786+0000","last_peered":"2024-09-10T22:35:44.661786+0000","last_clean":"2024-09-10T22:35:44.661786+0000","last_became_active":"2024-09-10T22:35:44.658672+0000","last_became_peered":"2024-09-10T22:35:44.658672+0000","last_unstale":"2024-09-10T22:35:44.661786+0000","last_undegraded":"2024-09-10T22:35:44.661786+0000","last_fullsized":"2024-09-10T22:35:44.661786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:17.176523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653565+0000","last_change":"2024-09-10T22:35:43.655647+0000","last_active":"2024-09-10T22:35:45.653565+0000","last_peered":"2024-09-10T22:35:45.653565+0000","last_clean":"2024-09-10T22:35:45.653565+0000","last_became_active":"2024-09-10T22:35:43.655346+0000","last_became_peered":"2024-09-10T22:35:43.655346+0000","last_unstale":"2024-09-10T22:35:45.653565+0000","last_undegraded":"2024-09-10T22:35:45.653565+0000","last_fullsized":"2024-09-10T22:35:45.653565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:16.171929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.665121+0000","last_change":"2024-09-10T22:35:44.662292+0000","last_active":"2024-09-10T22:35:44.665121+0000","last_peered":"2024-09-10T22:35:44.665121+0000","last_clean":"2024-09-10T22:35:44.665121+0000","last_became_active":"2024-09-10T22:35:44.662032+0000","last_became_peered":"2024-09-10T22:35:44.662032+0000","last_unstale":"2024-09-10T22:35:44.665121+0000","last_undegraded":"2024-09-10T22:35:44.665121+0000","last_fullsized":"2024-09-10T22:35:44.665121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:31.822145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.672112+0000","last_change":"2024-09-10T22:35:43.656104+0000","last_active":"2024-09-10T22:35:44.672112+0000","last_peered":"2024-09-10T22:35:44.672112+0000","last_clean":"2024-09-10T22:35:44.672112+0000","last_became_active":"2024-09-10T22:35:43.655798+0000","last_became_peered":"2024-09-10T22:35:43.655798+0000","last_unstale":"2024-09-10T22:35:44.672112+0000","last_undegraded":"2024-09-10T22:35:44.672112+0000","last_fullsized":"2024-09-10T22:35:44.672112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:48.438149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655033+0000","last_change":"2024-09-10T22:35:44.669098+0000","last_active":"2024-09-10T22:35:45.655033+0000","last_peered":"2024-09-10T22:35:45.655033+0000","last_clean":"2024-09-10T22:35:45.655033+0000","last_became_active":"2024-09-10T22:35:44.668486+0000","last_became_peered":"2024-09-10T22:35:44.668486+0000","last_unstale":"2024-09-10T22:35:45.655033+0000","last_undegraded":"2024-09-10T22:35:45.655033+0000","last_fullsized":"2024-09-10T22:35:45.655033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:01.236708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654728+0000","last_change":"2024-09-10T22:35:43.659628+0000","last_active":"2024-09-10T22:35:45.654728+0000","last_peered":"2024-09-10T22:35:45.654728+0000","last_clean":"2024-09-10T22:35:45.654728+0000","last_became_active":"2024-09-10T22:35:43.659308+0000","last_became_peered":"2024-09-10T22:35:43.659308+0000","last_unstale":"2024-09-10T22:35:45.654728+0000","last_undegraded":"2024-09-10T22:35:45.654728+0000","last_fullsized":"2024-09-10T22:35:45.654728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:22.440344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652896+0000","last_change":"2024-09-10T22:35:44.657571+0000","last_active":"2024-09-10T22:35:45.652896+0000","last_peered":"2024-09-10T22:35:45.652896+0000","last_clean":"2024-09-10T22:35:45.652896+0000","last_became_active":"2024-09-10T22:35:44.657300+0000","last_became_peered":"2024-09-10T22:35:44.657300+0000","last_unstale":"2024-09-10T22:35:45.652896+0000","last_undegraded":"2024-09-10T22:35:45.652896+0000","last_fullsized":"2024-09-10T22:35:45.652896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:54.836977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.650837+0000","last_change":"2024-09-10T22:35:43.655029+0000","last_active":"2024-09-10T22:35:44.650837+0000","last_peered":"2024-09-10T22:35:44.650837+0000","last_clean":"2024-09-10T22:35:44.650837+0000","last_became_active":"2024-09-10T22:35:43.654732+0000","last_became_peered":"2024-09-10T22:35:43.654732+0000","last_unstale":"2024-09-10T22:35:44.650837+0000","last_undegraded":"2024-09-10T22:35:44.650837+0000","last_fullsized":"2024-09-10T22:35:44.650837+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:45.095939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.657959+0000","last_change":"2024-09-10T22:35:44.655653+0000","last_active":"2024-09-10T22:35:44.657959+0000","last_peered":"2024-09-10T22:35:44.657959+0000","last_clean":"2024-09-10T22:35:44.657959+0000","last_became_active":"2024-09-10T22:35:44.655373+0000","last_became_peered":"2024-09-10T22:35:44.655373+0000","last_unstale":"2024-09-10T22:35:44.657959+0000","last_undegraded":"2024-09-10T22:35:44.657959+0000","last_fullsized":"2024-09-10T22:35:44.657959+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:02.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652896+0000","last_change":"2024-09-10T22:35:43.653478+0000","last_active":"2024-09-10T22:35:45.652896+0000","last_peered":"2024-09-10T22:35:45.652896+0000","last_clean":"2024-09-10T22:35:45.652896+0000","last_became_active":"2024-09-10T22:35:43.653135+0000","last_became_peered":"2024-09-10T22:35:43.653135+0000","last_unstale":"2024-09-10T22:35:45.652896+0000","last_undegraded":"2024-09-10T22:35:45.652896+0000","last_fullsized":"2024-09-10T22:35:45.652896+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:51.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652782+0000","last_change":"2024-09-10T22:35:44.655629+0000","last_active":"2024-09-10T22:35:45.652782+0000","last_peered":"2024-09-10T22:35:45.652782+0000","last_clean":"2024-09-10T22:35:45.652782+0000","last_became_active":"2024-09-10T22:35:44.654546+0000","last_became_peered":"2024-09-10T22:35:44.654546+0000","last_unstale":"2024-09-10T22:35:45.652782+0000","last_undegraded":"2024-09-10T22:35:45.652782+0000","last_fullsized":"2024-09-10T22:35:45.652782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:25.975720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651552+0000","last_change":"2024-09-10T22:35:43.655195+0000","last_active":"2024-09-10T22:35:44.651552+0000","last_peered":"2024-09-10T22:35:44.651552+0000","last_clean":"2024-09-10T22:35:44.651552+0000","last_became_active":"2024-09-10T22:35:43.654881+0000","last_became_peered":"2024-09-10T22:35:43.654881+0000","last_unstale":"2024-09-10T22:35:44.651552+0000","last_undegraded":"2024-09-10T22:35:44.651552+0000","last_fullsized":"2024-09-10T22:35:44.651552+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:10.021023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.675054+0000","last_change":"2024-09-10T22:35:44.672917+0000","last_active":"2024-09-10T22:35:44.675054+0000","last_peered":"2024-09-10T22:35:44.675054+0000","last_clean":"2024-09-10T22:35:44.675054+0000","last_became_active":"2024-09-10T22:35:44.672644+0000","last_became_peered":"2024-09-10T22:35:44.672644+0000","last_unstale":"2024-09-10T22:35:44.675054+0000","last_undegraded":"2024-09-10T22:35:44.675054+0000","last_fullsized":"2024-09-10T22:35:44.675054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:37.104143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654238+0000","last_change":"2024-09-10T22:35:43.652113+0000","last_active":"2024-09-10T22:35:45.654238+0000","last_peered":"2024-09-10T22:35:45.654238+0000","last_clean":"2024-09-10T22:35:45.654238+0000","last_became_active":"2024-09-10T22:35:43.649256+0000","last_became_peered":"2024-09-10T22:35:43.649256+0000","last_unstale":"2024-09-10T22:35:45.654238+0000","last_undegraded":"2024-09-10T22:35:45.654238+0000","last_fullsized":"2024-09-10T22:35:45.654238+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:01.441828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654154+0000","last_change":"2024-09-10T22:35:44.665221+0000","last_active":"2024-09-10T22:35:45.654154+0000","last_peered":"2024-09-10T22:35:45.654154+0000","last_clean":"2024-09-10T22:35:45.654154+0000","last_became_active":"2024-09-10T22:35:44.664952+0000","last_became_peered":"2024-09-10T22:35:44.664952+0000","last_unstale":"2024-09-10T22:35:45.654154+0000","last_undegraded":"2024-09-10T22:35:45.654154+0000","last_fullsized":"2024-09-10T22:35:45.654154+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:54.836977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.652314+0000","last_change":"2024-09-10T22:35:43.660238+0000","last_active":"2024-09-10T22:35:44.652314+0000","last_peered":"2024-09-10T22:35:44.652314+0000","last_clean":"2024-09-10T22:35:44.652314+0000","last_became_active":"2024-09-10T22:35:43.659954+0000","last_became_peered":"2024-09-10T22:35:43.659954+0000","last_unstale":"2024-09-10T22:35:44.652314+0000","last_undegraded":"2024-09-10T22:35:44.652314+0000","last_fullsized":"2024-09-10T22:35:44.652314+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:20.614603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663424+0000","last_change":"2024-09-10T22:35:44.660556+0000","last_active":"2024-09-10T22:35:44.663424+0000","last_peered":"2024-09-10T22:35:44.663424+0000","last_clean":"2024-09-10T22:35:44.663424+0000","last_became_active":"2024-09-10T22:35:44.660289+0000","last_became_peered":"2024-09-10T22:35:44.660289+0000","last_unstale":"2024-09-10T22:35:44.663424+0000","last_undegraded":"2024-09-10T22:35:44.663424+0000","last_fullsized":"2024-09-10T22:35:44.663424+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:33.230297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.647759+0000","last_change":"2024-09-10T22:35:43.654344+0000","last_active":"2024-09-10T22:35:44.647759+0000","last_peered":"2024-09-10T22:35:44.647759+0000","last_clean":"2024-09-10T22:35:44.647759+0000","last_became_active":"2024-09-10T22:35:43.654071+0000","last_became_peered":"2024-09-10T22:35:43.654071+0000","last_unstale":"2024-09-10T22:35:44.647759+0000","last_undegraded":"2024-09-10T22:35:44.647759+0000","last_fullsized":"2024-09-10T22:35:44.647759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:02.734433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":128,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651702+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:44.651702+0000","last_peered":"2024-09-10T22:35:44.651702+0000","last_clean":"2024-09-10T22:35:44.651702+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:44.651702+0000","last_undegraded":"2024-09-10T22:35:44.651702+0000","last_fullsized":"2024-09-10T22:35:44.651702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":937984,"data_stored":719671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":348160,"data_stored":129829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":360448,"data_stored":136185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":360448,"data_stored":135617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":725881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":356352,"data_stored":130847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":344064,"data_stored":129283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":356352,"data_stored":130847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:48.666 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:35:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 -- 172.21.15.66:0/1957253539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b21f0 msgr2=0x7fc6500b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 --2- 172.21.15.66:0/1957253539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b21f0 0x7fc6500b2660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc65008c8d0 tx=0x7fc64802a980 comp rx=0 tx=0).stop 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 -- 172.21.15.66:0/1957253539 shutdown_connections 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 --2- 172.21.15.66:0/1957253539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b21f0 0x7fc6500b2660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 --2- 172.21.15.66:0/1957253539 >> v2:172.21.15.94:3300/0 conn(0x7fc65015dff0 0x7fc6500d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 -- 172.21.15.66:0/1957253539 >> 172.21.15.66:0/1957253539 conn(0x7fc6500d3de0 msgr2=0x7fc6500d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 -- 172.21.15.66:0/1957253539 shutdown_connections 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.805+0000 7fc65519f640 1 -- 172.21.15.66:0/1957253539 wait complete. 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.806+0000 7fc65519f640 1 Processor -- start 2024-09-10T22:35:48.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.806+0000 7fc65519f640 1 -- start start 2024-09-10T22:35:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc65519f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b0970 0x7fc65015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc65519f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500b21f0 0x7fc65015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc65519f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc65015dff0 0x7fc65006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.810 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc65519f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500cfb60 con 0x7fc6500b0970 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc65519f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500cf9e0 con 0x7fc6500b21f0 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc65519f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500cfce0 con 0x7fc65015dff0 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc64ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b0970 0x7fc65015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc64ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b0970 0x7fc65015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52428/0 (socket says 172.21.15.66:52428) 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc64ffff640 1 -- 172.21.15.66:0/805283870 learned_addr learned my addr 172.21.15.66:0/805283870 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc65499e640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc65015dff0 0x7fc65006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.807+0000 7fc64f7fe640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500b21f0 0x7fc65015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64ffff640 1 -- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc65015dff0 msgr2=0x7fc65006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64ffff640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc65015dff0 0x7fc65006e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64ffff640 1 -- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500b21f0 msgr2=0x7fc65015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64ffff640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500b21f0 0x7fc65015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64ffff640 1 -- 172.21.15.66:0/805283870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6500cbf80 con 0x7fc6500b0970 2024-09-10T22:35:48.811 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64ffff640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b0970 0x7fc65015a570 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c970 tx=0x7fc64000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64cff9640 1 -- 172.21.15.66:0/805283870 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640007bb0 con 0x7fc6500b0970 2024-09-10T22:35:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64cff9640 1 -- 172.21.15.66:0/805283870 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc640007e40 con 0x7fc6500b0970 2024-09-10T22:35:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6500cdd50 con 0x7fc6500b0970 2024-09-10T22:35:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc64cff9640 1 -- 172.21.15.66:0/805283870 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64000f690 con 0x7fc6500b0970 2024-09-10T22:35:48.812 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.808+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6500b4950 con 0x7fc6500b0970 2024-09-10T22:35:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.809+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6500d0190 con 0x7fc6500b0970 2024-09-10T22:35:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.810+0000 7fc64cff9640 1 -- 172.21.15.66:0/805283870 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc640020030 con 0x7fc6500b0970 2024-09-10T22:35:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.810+0000 7fc64cff9640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc63c077f70 0x7fc63c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.810+0000 7fc64cff9640 1 -- 172.21.15.66:0/805283870 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc64009a060 con 0x7fc6500b0970 2024-09-10T22:35:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.810+0000 7fc64f7fe640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc63c077f70 0x7fc63c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:48.813 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.811+0000 7fc64f7fe640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc63c077f70 0x7fc63c07a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc65006f1e0 tx=0x7fc644007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:48.816 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.814+0000 7fc64cff9640 1 -- 172.21.15.66:0/805283870 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc640014030 con 0x7fc6500b0970 2024-09-10T22:35:48.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.994+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6500b6700 con 0x7fc63c077f70 2024-09-10T22:35:49.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:48.999+0000 7fc64cff9640 1 -- 172.21.15.66:0/805283870 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7fc6500b6700 con 0x7fc63c077f70 2024-09-10T22:35:49.004 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:49.005 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc63c077f70 msgr2=0x7fc63c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc63c077f70 0x7fc63c07a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc65006f1e0 tx=0x7fc644007e30 comp rx=0 tx=0).stop 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b0970 msgr2=0x7fc65015a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b0970 0x7fc65015a570 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c970 tx=0x7fc64000ce30 comp rx=0 tx=0).stop 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 shutdown_connections 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc63c077f70 0x7fc63c07a430 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc65015dff0 0x7fc65006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6500b21f0 0x7fc65015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 --2- 172.21.15.66:0/805283870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc6500b0970 0x7fc65015a570 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 >> 172.21.15.66:0/805283870 conn(0x7fc6500d3de0 msgr2=0x7fc6500b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 shutdown_connections 2024-09-10T22:35:49.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.002+0000 7fc65519f640 1 -- 172.21.15.66:0/805283870 wait complete. 2024-09-10T22:35:49.015 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-10T22:35:48.602668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240988,"kb_used_data":3916,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733732,"statfs":{"total":773094113280,"available":772847341568,"internally_reserved":0,"allocated":4009984,"data_stored":2238160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.967962"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661269+0000","last_change":"2024-09-10T22:35:44.658461+0000","last_active":"2024-09-10T22:35:44.661269+0000","last_peered":"2024-09-10T22:35:44.661269+0000","last_clean":"2024-09-10T22:35:44.661269+0000","last_became_active":"2024-09-10T22:35:44.658183+0000","last_became_peered":"2024-09-10T22:35:44.658183+0000","last_unstale":"2024-09-10T22:35:44.661269+0000","last_undegraded":"2024-09-10T22:35:44.661269+0000","last_fullsized":"2024-09-10T22:35:44.661269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:52.718822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670511+0000","last_change":"2024-09-10T22:35:44.661094+0000","last_active":"2024-09-10T22:35:44.670511+0000","last_peered":"2024-09-10T22:35:44.670511+0000","last_clean":"2024-09-10T22:35:44.670511+0000","last_became_active":"2024-09-10T22:35:44.659854+0000","last_became_peered":"2024-09-10T22:35:44.659854+0000","last_unstale":"2024-09-10T22:35:44.670511+0000","last_undegraded":"2024-09-10T22:35:44.670511+0000","last_fullsized":"2024-09-10T22:35:44.670511+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:01.410053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654686+0000","last_change":"2024-09-10T22:35:44.655880+0000","last_active":"2024-09-10T22:35:45.654686+0000","last_peered":"2024-09-10T22:35:45.654686+0000","last_clean":"2024-09-10T22:35:45.654686+0000","last_became_active":"2024-09-10T22:35:44.655601+0000","last_became_peered":"2024-09-10T22:35:44.655601+0000","last_unstale":"2024-09-10T22:35:45.654686+0000","last_undegraded":"2024-09-10T22:35:45.654686+0000","last_fullsized":"2024-09-10T22:35:45.654686+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:33.161604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669617+0000","last_change":"2024-09-10T22:35:44.664831+0000","last_active":"2024-09-10T22:35:44.669617+0000","last_peered":"2024-09-10T22:35:44.669617+0000","last_clean":"2024-09-10T22:35:44.669617+0000","last_became_active":"2024-09-10T22:35:44.664238+0000","last_became_peered":"2024-09-10T22:35:44.664238+0000","last_unstale":"2024-09-10T22:35:44.669617+0000","last_undegraded":"2024-09-10T22:35:44.669617+0000","last_fullsized":"2024-09-10T22:35:44.669617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:14.187410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670178+0000","last_change":"2024-09-10T22:35:44.666874+0000","last_active":"2024-09-10T22:35:44.670178+0000","last_peered":"2024-09-10T22:35:44.670178+0000","last_clean":"2024-09-10T22:35:44.670178+0000","last_became_active":"2024-09-10T22:35:44.666541+0000","last_became_peered":"2024-09-10T22:35:44.666541+0000","last_unstale":"2024-09-10T22:35:44.670178+0000","last_undegraded":"2024-09-10T22:35:44.670178+0000","last_fullsized":"2024-09-10T22:35:44.670178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:00.279279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670660+0000","last_change":"2024-09-10T22:35:44.663837+0000","last_active":"2024-09-10T22:35:44.670660+0000","last_peered":"2024-09-10T22:35:44.670660+0000","last_clean":"2024-09-10T22:35:44.670660+0000","last_became_active":"2024-09-10T22:35:44.660799+0000","last_became_peered":"2024-09-10T22:35:44.660799+0000","last_unstale":"2024-09-10T22:35:44.670660+0000","last_undegraded":"2024-09-10T22:35:44.670660+0000","last_fullsized":"2024-09-10T22:35:44.670660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:33.385136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661763+0000","last_change":"2024-09-10T22:35:44.659433+0000","last_active":"2024-09-10T22:35:44.661763+0000","last_peered":"2024-09-10T22:35:44.661763+0000","last_clean":"2024-09-10T22:35:44.661763+0000","last_became_active":"2024-09-10T22:35:44.659079+0000","last_became_peered":"2024-09-10T22:35:44.659079+0000","last_unstale":"2024-09-10T22:35:44.661763+0000","last_undegraded":"2024-09-10T22:35:44.661763+0000","last_fullsized":"2024-09-10T22:35:44.661763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:13.884564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666521+0000","last_change":"2024-09-10T22:35:44.662626+0000","last_active":"2024-09-10T22:35:44.666521+0000","last_peered":"2024-09-10T22:35:44.666521+0000","last_clean":"2024-09-10T22:35:44.666521+0000","last_became_active":"2024-09-10T22:35:44.662241+0000","last_became_peered":"2024-09-10T22:35:44.662241+0000","last_unstale":"2024-09-10T22:35:44.666521+0000","last_undegraded":"2024-09-10T22:35:44.666521+0000","last_fullsized":"2024-09-10T22:35:44.666521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:48.022021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655343+0000","last_change":"2024-09-10T22:35:44.657732+0000","last_active":"2024-09-10T22:35:45.655343+0000","last_peered":"2024-09-10T22:35:45.655343+0000","last_clean":"2024-09-10T22:35:45.655343+0000","last_became_active":"2024-09-10T22:35:44.655449+0000","last_became_peered":"2024-09-10T22:35:44.655449+0000","last_unstale":"2024-09-10T22:35:45.655343+0000","last_undegraded":"2024-09-10T22:35:45.655343+0000","last_fullsized":"2024-09-10T22:35:45.655343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:42.465443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.672409+0000","last_change":"2024-09-10T22:35:44.669432+0000","last_active":"2024-09-10T22:35:44.672409+0000","last_peered":"2024-09-10T22:35:44.672409+0000","last_clean":"2024-09-10T22:35:44.672409+0000","last_became_active":"2024-09-10T22:35:44.668907+0000","last_became_peered":"2024-09-10T22:35:44.668907+0000","last_unstale":"2024-09-10T22:35:44.672409+0000","last_undegraded":"2024-09-10T22:35:44.672409+0000","last_fullsized":"2024-09-10T22:35:44.672409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:50.077818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.660845+0000","last_change":"2024-09-10T22:35:44.658442+0000","last_active":"2024-09-10T22:35:44.660845+0000","last_peered":"2024-09-10T22:35:44.660845+0000","last_clean":"2024-09-10T22:35:44.660845+0000","last_became_active":"2024-09-10T22:35:44.658163+0000","last_became_peered":"2024-09-10T22:35:44.658163+0000","last_unstale":"2024-09-10T22:35:44.660845+0000","last_undegraded":"2024-09-10T22:35:44.660845+0000","last_fullsized":"2024-09-10T22:35:44.660845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:29.247536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654317+0000","last_change":"2024-09-10T22:35:44.665674+0000","last_active":"2024-09-10T22:35:45.654317+0000","last_peered":"2024-09-10T22:35:45.654317+0000","last_clean":"2024-09-10T22:35:45.654317+0000","last_became_active":"2024-09-10T22:35:44.665440+0000","last_became_peered":"2024-09-10T22:35:44.665440+0000","last_unstale":"2024-09-10T22:35:45.654317+0000","last_undegraded":"2024-09-10T22:35:45.654317+0000","last_fullsized":"2024-09-10T22:35:45.654317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:36.866074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670152+0000","last_change":"2024-09-10T22:35:44.665750+0000","last_active":"2024-09-10T22:35:44.670152+0000","last_peered":"2024-09-10T22:35:44.670152+0000","last_clean":"2024-09-10T22:35:44.670152+0000","last_became_active":"2024-09-10T22:35:44.665475+0000","last_became_peered":"2024-09-10T22:35:44.665475+0000","last_unstale":"2024-09-10T22:35:44.670152+0000","last_undegraded":"2024-09-10T22:35:44.670152+0000","last_fullsized":"2024-09-10T22:35:44.670152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:12.507140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670634+0000","last_change":"2024-09-10T22:35:44.660754+0000","last_active":"2024-09-10T22:35:44.670634+0000","last_peered":"2024-09-10T22:35:44.670634+0000","last_clean":"2024-09-10T22:35:44.670634+0000","last_became_active":"2024-09-10T22:35:44.660453+0000","last_became_peered":"2024-09-10T22:35:44.660453+0000","last_unstale":"2024-09-10T22:35:44.670634+0000","last_undegraded":"2024-09-10T22:35:44.670634+0000","last_fullsized":"2024-09-10T22:35:44.670634+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:22.577095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653915+0000","last_change":"2024-09-10T22:35:44.654084+0000","last_active":"2024-09-10T22:35:45.653915+0000","last_peered":"2024-09-10T22:35:45.653915+0000","last_clean":"2024-09-10T22:35:45.653915+0000","last_became_active":"2024-09-10T22:35:44.653681+0000","last_became_peered":"2024-09-10T22:35:44.653681+0000","last_unstale":"2024-09-10T22:35:45.653915+0000","last_undegraded":"2024-09-10T22:35:45.653915+0000","last_fullsized":"2024-09-10T22:35:45.653915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:25.907466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670003+0000","last_change":"2024-09-10T22:35:44.668702+0000","last_active":"2024-09-10T22:35:44.670003+0000","last_peered":"2024-09-10T22:35:44.670003+0000","last_clean":"2024-09-10T22:35:44.670003+0000","last_became_active":"2024-09-10T22:35:44.668405+0000","last_became_peered":"2024-09-10T22:35:44.668405+0000","last_unstale":"2024-09-10T22:35:44.670003+0000","last_undegraded":"2024-09-10T22:35:44.670003+0000","last_fullsized":"2024-09-10T22:35:44.670003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:42.465443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666460+0000","last_change":"2024-09-10T22:35:44.662295+0000","last_active":"2024-09-10T22:35:44.666460+0000","last_peered":"2024-09-10T22:35:44.666460+0000","last_clean":"2024-09-10T22:35:44.666460+0000","last_became_active":"2024-09-10T22:35:44.661967+0000","last_became_peered":"2024-09-10T22:35:44.661967+0000","last_unstale":"2024-09-10T22:35:44.666460+0000","last_undegraded":"2024-09-10T22:35:44.666460+0000","last_fullsized":"2024-09-10T22:35:44.666460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:14.941677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669489+0000","last_change":"2024-09-10T22:35:44.660421+0000","last_active":"2024-09-10T22:35:44.669489+0000","last_peered":"2024-09-10T22:35:44.669489+0000","last_clean":"2024-09-10T22:35:44.669489+0000","last_became_active":"2024-09-10T22:35:44.659490+0000","last_became_peered":"2024-09-10T22:35:44.659490+0000","last_unstale":"2024-09-10T22:35:44.669489+0000","last_undegraded":"2024-09-10T22:35:44.669489+0000","last_fullsized":"2024-09-10T22:35:44.669489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:50.476281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.675054+0000","last_change":"2024-09-10T22:35:44.672017+0000","last_active":"2024-09-10T22:35:44.675054+0000","last_peered":"2024-09-10T22:35:44.675054+0000","last_clean":"2024-09-10T22:35:44.675054+0000","last_became_active":"2024-09-10T22:35:44.671729+0000","last_became_peered":"2024-09-10T22:35:44.671729+0000","last_unstale":"2024-09-10T22:35:44.675054+0000","last_undegraded":"2024-09-10T22:35:44.675054+0000","last_fullsized":"2024-09-10T22:35:44.675054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:55.472544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.665189+0000","last_change":"2024-09-10T22:35:44.662346+0000","last_active":"2024-09-10T22:35:44.665189+0000","last_peered":"2024-09-10T22:35:44.665189+0000","last_clean":"2024-09-10T22:35:44.665189+0000","last_became_active":"2024-09-10T22:35:44.662064+0000","last_became_peered":"2024-09-10T22:35:44.662064+0000","last_unstale":"2024-09-10T22:35:44.665189+0000","last_undegraded":"2024-09-10T22:35:44.665189+0000","last_fullsized":"2024-09-10T22:35:44.665189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:25.361336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655299+0000","last_change":"2024-09-10T22:35:44.656870+0000","last_active":"2024-09-10T22:35:45.655299+0000","last_peered":"2024-09-10T22:35:45.655299+0000","last_clean":"2024-09-10T22:35:45.655299+0000","last_became_active":"2024-09-10T22:35:44.656109+0000","last_became_peered":"2024-09-10T22:35:44.656109+0000","last_unstale":"2024-09-10T22:35:45.655299+0000","last_undegraded":"2024-09-10T22:35:45.655299+0000","last_fullsized":"2024-09-10T22:35:45.655299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:01.275798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654873+0000","last_change":"2024-09-10T22:35:44.666261+0000","last_active":"2024-09-10T22:35:45.654873+0000","last_peered":"2024-09-10T22:35:45.654873+0000","last_clean":"2024-09-10T22:35:45.654873+0000","last_became_active":"2024-09-10T22:35:44.665965+0000","last_became_peered":"2024-09-10T22:35:44.665965+0000","last_unstale":"2024-09-10T22:35:45.654873+0000","last_undegraded":"2024-09-10T22:35:45.654873+0000","last_fullsized":"2024-09-10T22:35:45.654873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:15.559224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652660+0000","last_change":"2024-09-10T22:35:44.655485+0000","last_active":"2024-09-10T22:35:45.652660+0000","last_peered":"2024-09-10T22:35:45.652660+0000","last_clean":"2024-09-10T22:35:45.652660+0000","last_became_active":"2024-09-10T22:35:44.655176+0000","last_became_peered":"2024-09-10T22:35:44.655176+0000","last_unstale":"2024-09-10T22:35:45.652660+0000","last_undegraded":"2024-09-10T22:35:45.652660+0000","last_fullsized":"2024-09-10T22:35:45.652660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:02.543958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653681+0000","last_change":"2024-09-10T22:35:44.665023+0000","last_active":"2024-09-10T22:35:45.653681+0000","last_peered":"2024-09-10T22:35:45.653681+0000","last_clean":"2024-09-10T22:35:45.653681+0000","last_became_active":"2024-09-10T22:35:44.664755+0000","last_became_peered":"2024-09-10T22:35:44.664755+0000","last_unstale":"2024-09-10T22:35:45.653681+0000","last_undegraded":"2024-09-10T22:35:45.653681+0000","last_fullsized":"2024-09-10T22:35:45.653681+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:11.295729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669684+0000","last_change":"2024-09-10T22:35:44.658308+0000","last_active":"2024-09-10T22:35:44.669684+0000","last_peered":"2024-09-10T22:35:44.669684+0000","last_clean":"2024-09-10T22:35:44.669684+0000","last_became_active":"2024-09-10T22:35:44.657751+0000","last_became_peered":"2024-09-10T22:35:44.657751+0000","last_unstale":"2024-09-10T22:35:44.669684+0000","last_undegraded":"2024-09-10T22:35:44.669684+0000","last_fullsized":"2024-09-10T22:35:44.669684+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:46.030439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669462+0000","last_change":"2024-09-10T22:35:44.660112+0000","last_active":"2024-09-10T22:35:44.669462+0000","last_peered":"2024-09-10T22:35:44.669462+0000","last_clean":"2024-09-10T22:35:44.669462+0000","last_became_active":"2024-09-10T22:35:44.659148+0000","last_became_peered":"2024-09-10T22:35:44.659148+0000","last_unstale":"2024-09-10T22:35:44.669462+0000","last_undegraded":"2024-09-10T22:35:44.669462+0000","last_fullsized":"2024-09-10T22:35:44.669462+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:54.559529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669628+0000","last_change":"2024-09-10T22:35:44.664434+0000","last_active":"2024-09-10T22:35:44.669628+0000","last_peered":"2024-09-10T22:35:44.669628+0000","last_clean":"2024-09-10T22:35:44.669628+0000","last_became_active":"2024-09-10T22:35:44.663532+0000","last_became_peered":"2024-09-10T22:35:44.663532+0000","last_unstale":"2024-09-10T22:35:44.669628+0000","last_undegraded":"2024-09-10T22:35:44.669628+0000","last_fullsized":"2024-09-10T22:35:44.669628+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:51.866302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654570+0000","last_change":"2024-09-10T22:35:44.652979+0000","last_active":"2024-09-10T22:35:45.654570+0000","last_peered":"2024-09-10T22:35:45.654570+0000","last_clean":"2024-09-10T22:35:45.654570+0000","last_became_active":"2024-09-10T22:35:44.652607+0000","last_became_peered":"2024-09-10T22:35:44.652607+0000","last_unstale":"2024-09-10T22:35:45.654570+0000","last_undegraded":"2024-09-10T22:35:45.654570+0000","last_fullsized":"2024-09-10T22:35:45.654570+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:24.032728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655317+0000","last_change":"2024-09-10T22:35:44.666266+0000","last_active":"2024-09-10T22:35:45.655317+0000","last_peered":"2024-09-10T22:35:45.655317+0000","last_clean":"2024-09-10T22:35:45.655317+0000","last_became_active":"2024-09-10T22:35:44.665974+0000","last_became_peered":"2024-09-10T22:35:44.665974+0000","last_unstale":"2024-09-10T22:35:45.655317+0000","last_undegraded":"2024-09-10T22:35:45.655317+0000","last_fullsized":"2024-09-10T22:35:45.655317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:20.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661698+0000","last_change":"2024-09-10T22:35:44.659157+0000","last_active":"2024-09-10T22:35:44.661698+0000","last_peered":"2024-09-10T22:35:44.661698+0000","last_clean":"2024-09-10T22:35:44.661698+0000","last_became_active":"2024-09-10T22:35:44.658863+0000","last_became_peered":"2024-09-10T22:35:44.658863+0000","last_unstale":"2024-09-10T22:35:44.661698+0000","last_undegraded":"2024-09-10T22:35:44.661698+0000","last_fullsized":"2024-09-10T22:35:44.661698+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:59.418237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.662298+0000","last_change":"2024-09-10T22:35:44.660953+0000","last_active":"2024-09-10T22:35:44.662298+0000","last_peered":"2024-09-10T22:35:44.662298+0000","last_clean":"2024-09-10T22:35:44.662298+0000","last_became_active":"2024-09-10T22:35:44.660694+0000","last_became_peered":"2024-09-10T22:35:44.660694+0000","last_unstale":"2024-09-10T22:35:44.662298+0000","last_undegraded":"2024-09-10T22:35:44.662298+0000","last_fullsized":"2024-09-10T22:35:44.662298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:59.131318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666499+0000","last_change":"2024-09-10T22:35:44.662495+0000","last_active":"2024-09-10T22:35:44.666499+0000","last_peered":"2024-09-10T22:35:44.666499+0000","last_clean":"2024-09-10T22:35:44.666499+0000","last_became_active":"2024-09-10T22:35:44.661887+0000","last_became_peered":"2024-09-10T22:35:44.661887+0000","last_unstale":"2024-09-10T22:35:44.666499+0000","last_undegraded":"2024-09-10T22:35:44.666499+0000","last_fullsized":"2024-09-10T22:35:44.666499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:19.759120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652879+0000","last_change":"2024-09-10T22:35:44.656320+0000","last_active":"2024-09-10T22:35:45.652879+0000","last_peered":"2024-09-10T22:35:45.652879+0000","last_clean":"2024-09-10T22:35:45.652879+0000","last_became_active":"2024-09-10T22:35:44.655990+0000","last_became_peered":"2024-09-10T22:35:44.655990+0000","last_unstale":"2024-09-10T22:35:45.652879+0000","last_undegraded":"2024-09-10T22:35:45.652879+0000","last_fullsized":"2024-09-10T22:35:45.652879+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:12.936488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670070+0000","last_change":"2024-09-10T22:35:43.650422+0000","last_active":"2024-09-10T22:35:44.670070+0000","last_peered":"2024-09-10T22:35:44.670070+0000","last_clean":"2024-09-10T22:35:44.670070+0000","last_became_active":"2024-09-10T22:35:43.650163+0000","last_became_peered":"2024-09-10T22:35:43.650163+0000","last_unstale":"2024-09-10T22:35:44.670070+0000","last_undegraded":"2024-09-10T22:35:44.670070+0000","last_fullsized":"2024-09-10T22:35:44.670070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:46.829650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661291+0000","last_change":"2024-09-10T22:35:44.659187+0000","last_active":"2024-09-10T22:35:44.661291+0000","last_peered":"2024-09-10T22:35:44.661291+0000","last_clean":"2024-09-10T22:35:44.661291+0000","last_became_active":"2024-09-10T22:35:44.658902+0000","last_became_peered":"2024-09-10T22:35:44.658902+0000","last_unstale":"2024-09-10T22:35:44.661291+0000","last_undegraded":"2024-09-10T22:35:44.661291+0000","last_fullsized":"2024-09-10T22:35:44.661291+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.095500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652805+0000","last_change":"2024-09-10T22:35:43.656995+0000","last_active":"2024-09-10T22:35:45.652805+0000","last_peered":"2024-09-10T22:35:45.652805+0000","last_clean":"2024-09-10T22:35:45.652805+0000","last_became_active":"2024-09-10T22:35:43.656678+0000","last_became_peered":"2024-09-10T22:35:43.656678+0000","last_unstale":"2024-09-10T22:35:45.652805+0000","last_undegraded":"2024-09-10T22:35:45.652805+0000","last_fullsized":"2024-09-10T22:35:45.652805+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:01.441828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655357+0000","last_change":"2024-09-10T22:35:44.664813+0000","last_active":"2024-09-10T22:35:45.655357+0000","last_peered":"2024-09-10T22:35:45.655357+0000","last_clean":"2024-09-10T22:35:45.655357+0000","last_became_active":"2024-09-10T22:35:44.664482+0000","last_became_peered":"2024-09-10T22:35:44.664482+0000","last_unstale":"2024-09-10T22:35:45.655357+0000","last_undegraded":"2024-09-10T22:35:45.655357+0000","last_fullsized":"2024-09-10T22:35:45.655357+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:48.022021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.673898+0000","last_change":"2024-09-10T22:35:43.663727+0000","last_active":"2024-09-10T22:35:44.673898+0000","last_peered":"2024-09-10T22:35:44.673898+0000","last_clean":"2024-09-10T22:35:44.673898+0000","last_became_active":"2024-09-10T22:35:43.663418+0000","last_became_peered":"2024-09-10T22:35:43.663418+0000","last_unstale":"2024-09-10T22:35:44.673898+0000","last_undegraded":"2024-09-10T22:35:44.673898+0000","last_fullsized":"2024-09-10T22:35:44.673898+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:21.593943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669601+0000","last_change":"2024-09-10T22:35:44.658719+0000","last_active":"2024-09-10T22:35:44.669601+0000","last_peered":"2024-09-10T22:35:44.669601+0000","last_clean":"2024-09-10T22:35:44.669601+0000","last_became_active":"2024-09-10T22:35:44.658397+0000","last_became_peered":"2024-09-10T22:35:44.658397+0000","last_unstale":"2024-09-10T22:35:44.669601+0000","last_undegraded":"2024-09-10T22:35:44.669601+0000","last_fullsized":"2024-09-10T22:35:44.669601+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:49:53.725746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.671442+0000","last_change":"2024-09-10T22:35:43.656397+0000","last_active":"2024-09-10T22:35:44.671442+0000","last_peered":"2024-09-10T22:35:44.671442+0000","last_clean":"2024-09-10T22:35:44.671442+0000","last_became_active":"2024-09-10T22:35:43.656125+0000","last_became_peered":"2024-09-10T22:35:43.656125+0000","last_unstale":"2024-09-10T22:35:44.671442+0000","last_undegraded":"2024-09-10T22:35:44.671442+0000","last_fullsized":"2024-09-10T22:35:44.671442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:55.949105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663373+0000","last_change":"2024-09-10T22:35:44.659872+0000","last_active":"2024-09-10T22:35:44.663373+0000","last_peered":"2024-09-10T22:35:44.663373+0000","last_clean":"2024-09-10T22:35:44.663373+0000","last_became_active":"2024-09-10T22:35:44.659602+0000","last_became_peered":"2024-09-10T22:35:44.659602+0000","last_unstale":"2024-09-10T22:35:44.663373+0000","last_undegraded":"2024-09-10T22:35:44.663373+0000","last_fullsized":"2024-09-10T22:35:44.663373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:20.838308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655433+0000","last_change":"2024-09-10T22:35:43.659132+0000","last_active":"2024-09-10T22:35:45.655433+0000","last_peered":"2024-09-10T22:35:45.655433+0000","last_clean":"2024-09-10T22:35:45.655433+0000","last_became_active":"2024-09-10T22:35:43.658844+0000","last_became_peered":"2024-09-10T22:35:43.658844+0000","last_unstale":"2024-09-10T22:35:45.655433+0000","last_undegraded":"2024-09-10T22:35:45.655433+0000","last_fullsized":"2024-09-10T22:35:45.655433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:02.734433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654653+0000","last_change":"2024-09-10T22:35:44.648187+0000","last_active":"2024-09-10T22:35:45.654653+0000","last_peered":"2024-09-10T22:35:45.654653+0000","last_clean":"2024-09-10T22:35:45.654653+0000","last_became_active":"2024-09-10T22:35:44.647909+0000","last_became_peered":"2024-09-10T22:35:44.647909+0000","last_unstale":"2024-09-10T22:35:45.654653+0000","last_undegraded":"2024-09-10T22:35:45.654653+0000","last_fullsized":"2024-09-10T22:35:45.654653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:15.818568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655023+0000","last_change":"2024-09-10T22:35:43.657036+0000","last_active":"2024-09-10T22:35:45.655023+0000","last_peered":"2024-09-10T22:35:45.655023+0000","last_clean":"2024-09-10T22:35:45.655023+0000","last_became_active":"2024-09-10T22:35:43.656794+0000","last_became_peered":"2024-09-10T22:35:43.656794+0000","last_unstale":"2024-09-10T22:35:45.655023+0000","last_undegraded":"2024-09-10T22:35:45.655023+0000","last_fullsized":"2024-09-10T22:35:45.655023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:40:02.923542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653981+0000","last_change":"2024-09-10T22:35:44.655450+0000","last_active":"2024-09-10T22:35:45.653981+0000","last_peered":"2024-09-10T22:35:45.653981+0000","last_clean":"2024-09-10T22:35:45.653981+0000","last_became_active":"2024-09-10T22:35:44.655177+0000","last_became_peered":"2024-09-10T22:35:44.655177+0000","last_unstale":"2024-09-10T22:35:45.653981+0000","last_undegraded":"2024-09-10T22:35:45.653981+0000","last_fullsized":"2024-09-10T22:35:45.653981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:12.936488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.649035+0000","last_change":"2024-09-10T22:35:43.659764+0000","last_active":"2024-09-10T22:35:44.649035+0000","last_peered":"2024-09-10T22:35:44.649035+0000","last_clean":"2024-09-10T22:35:44.649035+0000","last_became_active":"2024-09-10T22:35:43.659427+0000","last_became_peered":"2024-09-10T22:35:43.659427+0000","last_unstale":"2024-09-10T22:35:44.649035+0000","last_undegraded":"2024-09-10T22:35:44.649035+0000","last_fullsized":"2024-09-10T22:35:44.649035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:58.208840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670504+0000","last_change":"2024-09-10T22:35:44.663062+0000","last_active":"2024-09-10T22:35:44.670504+0000","last_peered":"2024-09-10T22:35:44.670504+0000","last_clean":"2024-09-10T22:35:44.670504+0000","last_became_active":"2024-09-10T22:35:44.662797+0000","last_became_peered":"2024-09-10T22:35:44.662797+0000","last_unstale":"2024-09-10T22:35:44.670504+0000","last_undegraded":"2024-09-10T22:35:44.670504+0000","last_fullsized":"2024-09-10T22:35:44.670504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:46.161284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653970+0000","last_change":"2024-09-10T22:35:43.649850+0000","last_active":"2024-09-10T22:35:45.653970+0000","last_peered":"2024-09-10T22:35:45.653970+0000","last_clean":"2024-09-10T22:35:45.653970+0000","last_became_active":"2024-09-10T22:35:43.649556+0000","last_became_peered":"2024-09-10T22:35:43.649556+0000","last_unstale":"2024-09-10T22:35:45.653970+0000","last_undegraded":"2024-09-10T22:35:45.653970+0000","last_fullsized":"2024-09-10T22:35:45.653970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:53.970339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654921+0000","last_change":"2024-09-10T22:35:44.671682+0000","last_active":"2024-09-10T22:35:45.654921+0000","last_peered":"2024-09-10T22:35:45.654921+0000","last_clean":"2024-09-10T22:35:45.654921+0000","last_became_active":"2024-09-10T22:35:44.671387+0000","last_became_peered":"2024-09-10T22:35:44.671387+0000","last_unstale":"2024-09-10T22:35:45.654921+0000","last_undegraded":"2024-09-10T22:35:45.654921+0000","last_fullsized":"2024-09-10T22:35:45.654921+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:28.996396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.654590+0000","last_change":"2024-09-10T22:35:43.661676+0000","last_active":"2024-09-10T22:35:44.654590+0000","last_peered":"2024-09-10T22:35:44.654590+0000","last_clean":"2024-09-10T22:35:44.654590+0000","last_became_active":"2024-09-10T22:35:43.661191+0000","last_became_peered":"2024-09-10T22:35:43.661191+0000","last_unstale":"2024-09-10T22:35:44.654590+0000","last_undegraded":"2024-09-10T22:35:44.654590+0000","last_fullsized":"2024-09-10T22:35:44.654590+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:39.996112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.675010+0000","last_change":"2024-09-10T22:35:44.672583+0000","last_active":"2024-09-10T22:35:44.675010+0000","last_peered":"2024-09-10T22:35:44.675010+0000","last_clean":"2024-09-10T22:35:44.675010+0000","last_became_active":"2024-09-10T22:35:44.672310+0000","last_became_peered":"2024-09-10T22:35:44.672310+0000","last_unstale":"2024-09-10T22:35:44.675010+0000","last_undegraded":"2024-09-10T22:35:44.675010+0000","last_fullsized":"2024-09-10T22:35:44.675010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:13.748432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652906+0000","last_change":"2024-09-10T22:35:43.661493+0000","last_active":"2024-09-10T22:35:45.652906+0000","last_peered":"2024-09-10T22:35:45.652906+0000","last_clean":"2024-09-10T22:35:45.652906+0000","last_became_active":"2024-09-10T22:35:43.661224+0000","last_became_peered":"2024-09-10T22:35:43.661224+0000","last_unstale":"2024-09-10T22:35:45.652906+0000","last_undegraded":"2024-09-10T22:35:45.652906+0000","last_fullsized":"2024-09-10T22:35:45.652906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:19.090906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670417+0000","last_change":"2024-09-10T22:35:44.661923+0000","last_active":"2024-09-10T22:35:44.670417+0000","last_peered":"2024-09-10T22:35:44.670417+0000","last_clean":"2024-09-10T22:35:44.670417+0000","last_became_active":"2024-09-10T22:35:44.661622+0000","last_became_peered":"2024-09-10T22:35:44.661622+0000","last_unstale":"2024-09-10T22:35:44.670417+0000","last_undegraded":"2024-09-10T22:35:44.670417+0000","last_fullsized":"2024-09-10T22:35:44.670417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:57.051422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654830+0000","last_change":"2024-09-10T22:35:43.657210+0000","last_active":"2024-09-10T22:35:45.654830+0000","last_peered":"2024-09-10T22:35:45.654830+0000","last_clean":"2024-09-10T22:35:45.654830+0000","last_became_active":"2024-09-10T22:35:43.656950+0000","last_became_peered":"2024-09-10T22:35:43.656950+0000","last_unstale":"2024-09-10T22:35:45.654830+0000","last_undegraded":"2024-09-10T22:35:45.654830+0000","last_fullsized":"2024-09-10T22:35:45.654830+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:12.879969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654998+0000","last_change":"2024-09-10T22:35:44.671824+0000","last_active":"2024-09-10T22:35:45.654998+0000","last_peered":"2024-09-10T22:35:45.654998+0000","last_clean":"2024-09-10T22:35:45.654998+0000","last_became_active":"2024-09-10T22:35:44.671590+0000","last_became_peered":"2024-09-10T22:35:44.671590+0000","last_unstale":"2024-09-10T22:35:45.654998+0000","last_undegraded":"2024-09-10T22:35:45.654998+0000","last_fullsized":"2024-09-10T22:35:45.654998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:46.220293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.676456+0000","last_change":"2024-09-10T22:35:43.663926+0000","last_active":"2024-09-10T22:35:44.676456+0000","last_peered":"2024-09-10T22:35:44.676456+0000","last_clean":"2024-09-10T22:35:44.676456+0000","last_became_active":"2024-09-10T22:35:43.663615+0000","last_became_peered":"2024-09-10T22:35:43.663615+0000","last_unstale":"2024-09-10T22:35:44.676456+0000","last_undegraded":"2024-09-10T22:35:44.676456+0000","last_fullsized":"2024-09-10T22:35:44.676456+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:12.306113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670634+0000","last_change":"2024-09-10T22:35:44.666504+0000","last_active":"2024-09-10T22:35:44.670634+0000","last_peered":"2024-09-10T22:35:44.670634+0000","last_clean":"2024-09-10T22:35:44.670634+0000","last_became_active":"2024-09-10T22:35:44.666149+0000","last_became_peered":"2024-09-10T22:35:44.666149+0000","last_unstale":"2024-09-10T22:35:44.670634+0000","last_undegraded":"2024-09-10T22:35:44.670634+0000","last_fullsized":"2024-09-10T22:35:44.670634+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:09.758004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655557+0000","last_change":"2024-09-10T22:35:43.654467+0000","last_active":"2024-09-10T22:35:45.655557+0000","last_peered":"2024-09-10T22:35:45.655557+0000","last_clean":"2024-09-10T22:35:45.655557+0000","last_became_active":"2024-09-10T22:35:43.653474+0000","last_became_peered":"2024-09-10T22:35:43.653474+0000","last_unstale":"2024-09-10T22:35:45.655557+0000","last_undegraded":"2024-09-10T22:35:45.655557+0000","last_fullsized":"2024-09-10T22:35:45.655557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:47.327430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.670757+0000","last_change":"2024-09-10T22:35:44.668271+0000","last_active":"2024-09-10T22:35:44.670757+0000","last_peered":"2024-09-10T22:35:44.670757+0000","last_clean":"2024-09-10T22:35:44.670757+0000","last_became_active":"2024-09-10T22:35:44.668001+0000","last_became_peered":"2024-09-10T22:35:44.668001+0000","last_unstale":"2024-09-10T22:35:44.670757+0000","last_undegraded":"2024-09-10T22:35:44.670757+0000","last_fullsized":"2024-09-10T22:35:44.670757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:38.801166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651424+0000","last_change":"2024-09-10T22:35:43.657001+0000","last_active":"2024-09-10T22:35:44.651424+0000","last_peered":"2024-09-10T22:35:44.651424+0000","last_clean":"2024-09-10T22:35:44.651424+0000","last_became_active":"2024-09-10T22:35:43.656698+0000","last_became_peered":"2024-09-10T22:35:43.656698+0000","last_unstale":"2024-09-10T22:35:44.651424+0000","last_undegraded":"2024-09-10T22:35:44.651424+0000","last_fullsized":"2024-09-10T22:35:44.651424+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:28.894829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.666540+0000","last_change":"2024-09-10T22:35:44.662824+0000","last_active":"2024-09-10T22:35:44.666540+0000","last_peered":"2024-09-10T22:35:44.666540+0000","last_clean":"2024-09-10T22:35:44.666540+0000","last_became_active":"2024-09-10T22:35:44.661179+0000","last_became_peered":"2024-09-10T22:35:44.661179+0000","last_unstale":"2024-09-10T22:35:44.666540+0000","last_undegraded":"2024-09-10T22:35:44.666540+0000","last_fullsized":"2024-09-10T22:35:44.666540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:20.087912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.649246+0000","last_change":"2024-09-10T22:35:43.661113+0000","last_active":"2024-09-10T22:35:44.649246+0000","last_peered":"2024-09-10T22:35:44.649246+0000","last_clean":"2024-09-10T22:35:44.649246+0000","last_became_active":"2024-09-10T22:35:43.660808+0000","last_became_peered":"2024-09-10T22:35:43.660808+0000","last_unstale":"2024-09-10T22:35:44.649246+0000","last_undegraded":"2024-09-10T22:35:44.649246+0000","last_fullsized":"2024-09-10T22:35:44.649246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:26.752833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661362+0000","last_change":"2024-09-10T22:35:44.660082+0000","last_active":"2024-09-10T22:35:44.661362+0000","last_peered":"2024-09-10T22:35:44.661362+0000","last_clean":"2024-09-10T22:35:44.661362+0000","last_became_active":"2024-09-10T22:35:44.659694+0000","last_became_peered":"2024-09-10T22:35:44.659694+0000","last_unstale":"2024-09-10T22:35:44.661362+0000","last_undegraded":"2024-09-10T22:35:44.661362+0000","last_fullsized":"2024-09-10T22:35:44.661362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:23.444938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.674657+0000","last_change":"2024-09-10T22:35:43.663130+0000","last_active":"2024-09-10T22:35:44.674657+0000","last_peered":"2024-09-10T22:35:44.674657+0000","last_clean":"2024-09-10T22:35:44.674657+0000","last_became_active":"2024-09-10T22:35:43.662833+0000","last_became_peered":"2024-09-10T22:35:43.662833+0000","last_unstale":"2024-09-10T22:35:44.674657+0000","last_undegraded":"2024-09-10T22:35:44.674657+0000","last_fullsized":"2024-09-10T22:35:44.674657+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:01.801641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655003+0000","last_change":"2024-09-10T22:35:44.654315+0000","last_active":"2024-09-10T22:35:45.655003+0000","last_peered":"2024-09-10T22:35:45.655003+0000","last_clean":"2024-09-10T22:35:45.655003+0000","last_became_active":"2024-09-10T22:35:44.654040+0000","last_became_peered":"2024-09-10T22:35:44.654040+0000","last_unstale":"2024-09-10T22:35:45.655003+0000","last_undegraded":"2024-09-10T22:35:45.655003+0000","last_fullsized":"2024-09-10T22:35:45.655003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:43.324344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.674597+0000","last_change":"2024-09-10T22:35:43.661835+0000","last_active":"2024-09-10T22:35:44.674597+0000","last_peered":"2024-09-10T22:35:44.674597+0000","last_clean":"2024-09-10T22:35:44.674597+0000","last_became_active":"2024-09-10T22:35:43.661517+0000","last_became_peered":"2024-09-10T22:35:43.661517+0000","last_unstale":"2024-09-10T22:35:44.674597+0000","last_undegraded":"2024-09-10T22:35:44.674597+0000","last_fullsized":"2024-09-10T22:35:44.674597+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:14.571477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654590+0000","last_change":"2024-09-10T22:35:44.661067+0000","last_active":"2024-09-10T22:35:45.654590+0000","last_peered":"2024-09-10T22:35:45.654590+0000","last_clean":"2024-09-10T22:35:45.654590+0000","last_became_active":"2024-09-10T22:35:44.660755+0000","last_became_peered":"2024-09-10T22:35:44.660755+0000","last_unstale":"2024-09-10T22:35:45.654590+0000","last_undegraded":"2024-09-10T22:35:45.654590+0000","last_fullsized":"2024-09-10T22:35:45.654590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:10.408250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655131+0000","last_change":"2024-09-10T22:35:43.650404+0000","last_active":"2024-09-10T22:35:45.655131+0000","last_peered":"2024-09-10T22:35:45.655131+0000","last_clean":"2024-09-10T22:35:45.655131+0000","last_became_active":"2024-09-10T22:35:43.650111+0000","last_became_peered":"2024-09-10T22:35:43.650111+0000","last_unstale":"2024-09-10T22:35:45.655131+0000","last_undegraded":"2024-09-10T22:35:45.655131+0000","last_fullsized":"2024-09-10T22:35:45.655131+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:37.492342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.669590+0000","last_change":"2024-09-10T22:35:44.664534+0000","last_active":"2024-09-10T22:35:44.669590+0000","last_peered":"2024-09-10T22:35:44.669590+0000","last_clean":"2024-09-10T22:35:44.669590+0000","last_became_active":"2024-09-10T22:35:44.663897+0000","last_became_peered":"2024-09-10T22:35:44.663897+0000","last_unstale":"2024-09-10T22:35:44.669590+0000","last_undegraded":"2024-09-10T22:35:44.669590+0000","last_fullsized":"2024-09-10T22:35:44.669590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:57.268790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655565+0000","last_change":"2024-09-10T22:35:43.650715+0000","last_active":"2024-09-10T22:35:45.655565+0000","last_peered":"2024-09-10T22:35:45.655565+0000","last_clean":"2024-09-10T22:35:45.655565+0000","last_became_active":"2024-09-10T22:35:43.650412+0000","last_became_peered":"2024-09-10T22:35:43.650412+0000","last_unstale":"2024-09-10T22:35:45.655565+0000","last_undegraded":"2024-09-10T22:35:45.655565+0000","last_fullsized":"2024-09-10T22:35:45.655565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:05.509208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663287+0000","last_change":"2024-09-10T22:35:44.660874+0000","last_active":"2024-09-10T22:35:44.663287+0000","last_peered":"2024-09-10T22:35:44.663287+0000","last_clean":"2024-09-10T22:35:44.663287+0000","last_became_active":"2024-09-10T22:35:44.659700+0000","last_became_peered":"2024-09-10T22:35:44.659700+0000","last_unstale":"2024-09-10T22:35:44.663287+0000","last_undegraded":"2024-09-10T22:35:44.663287+0000","last_fullsized":"2024-09-10T22:35:44.663287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:45.580791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655301+0000","last_change":"2024-09-10T22:35:43.659284+0000","last_active":"2024-09-10T22:35:45.655301+0000","last_peered":"2024-09-10T22:35:45.655301+0000","last_clean":"2024-09-10T22:35:45.655301+0000","last_became_active":"2024-09-10T22:35:43.659001+0000","last_became_peered":"2024-09-10T22:35:43.659001+0000","last_unstale":"2024-09-10T22:35:45.655301+0000","last_undegraded":"2024-09-10T22:35:45.655301+0000","last_fullsized":"2024-09-10T22:35:45.655301+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:51.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663311+0000","last_change":"2024-09-10T22:35:44.660625+0000","last_active":"2024-09-10T22:35:44.663311+0000","last_peered":"2024-09-10T22:35:44.663311+0000","last_clean":"2024-09-10T22:35:44.663311+0000","last_became_active":"2024-09-10T22:35:44.660325+0000","last_became_peered":"2024-09-10T22:35:44.660325+0000","last_unstale":"2024-09-10T22:35:44.663311+0000","last_undegraded":"2024-09-10T22:35:44.663311+0000","last_fullsized":"2024-09-10T22:35:44.663311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:19.135367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652962+0000","last_change":"2024-09-10T22:35:43.659599+0000","last_active":"2024-09-10T22:35:45.652962+0000","last_peered":"2024-09-10T22:35:45.652962+0000","last_clean":"2024-09-10T22:35:45.652962+0000","last_became_active":"2024-09-10T22:35:43.659258+0000","last_became_peered":"2024-09-10T22:35:43.659258+0000","last_unstale":"2024-09-10T22:35:45.652962+0000","last_undegraded":"2024-09-10T22:35:45.652962+0000","last_fullsized":"2024-09-10T22:35:45.652962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:47.327430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663244+0000","last_change":"2024-09-10T22:35:44.659463+0000","last_active":"2024-09-10T22:35:44.663244+0000","last_peered":"2024-09-10T22:35:44.663244+0000","last_clean":"2024-09-10T22:35:44.663244+0000","last_became_active":"2024-09-10T22:35:44.659151+0000","last_became_peered":"2024-09-10T22:35:44.659151+0000","last_unstale":"2024-09-10T22:35:44.663244+0000","last_undegraded":"2024-09-10T22:35:44.663244+0000","last_fullsized":"2024-09-10T22:35:44.663244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:02.818642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651565+0000","last_change":"2024-09-10T22:35:43.659671+0000","last_active":"2024-09-10T22:35:44.651565+0000","last_peered":"2024-09-10T22:35:44.651565+0000","last_clean":"2024-09-10T22:35:44.651565+0000","last_became_active":"2024-09-10T22:35:43.659273+0000","last_became_peered":"2024-09-10T22:35:43.659273+0000","last_unstale":"2024-09-10T22:35:44.651565+0000","last_undegraded":"2024-09-10T22:35:44.651565+0000","last_fullsized":"2024-09-10T22:35:44.651565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:47.327430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663661+0000","last_change":"2024-09-10T22:35:44.660247+0000","last_active":"2024-09-10T22:35:44.663661+0000","last_peered":"2024-09-10T22:35:44.663661+0000","last_clean":"2024-09-10T22:35:44.663661+0000","last_became_active":"2024-09-10T22:35:44.659974+0000","last_became_peered":"2024-09-10T22:35:44.659974+0000","last_unstale":"2024-09-10T22:35:44.663661+0000","last_undegraded":"2024-09-10T22:35:44.663661+0000","last_fullsized":"2024-09-10T22:35:44.663661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:54:08.855764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654137+0000","last_change":"2024-09-10T22:35:43.656257+0000","last_active":"2024-09-10T22:35:45.654137+0000","last_peered":"2024-09-10T22:35:45.654137+0000","last_clean":"2024-09-10T22:35:45.654137+0000","last_became_active":"2024-09-10T22:35:43.655985+0000","last_became_peered":"2024-09-10T22:35:43.655985+0000","last_unstale":"2024-09-10T22:35:45.654137+0000","last_undegraded":"2024-09-10T22:35:45.654137+0000","last_fullsized":"2024-09-10T22:35:45.654137+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:46.829650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.661786+0000","last_change":"2024-09-10T22:35:44.659257+0000","last_active":"2024-09-10T22:35:44.661786+0000","last_peered":"2024-09-10T22:35:44.661786+0000","last_clean":"2024-09-10T22:35:44.661786+0000","last_became_active":"2024-09-10T22:35:44.658672+0000","last_became_peered":"2024-09-10T22:35:44.658672+0000","last_unstale":"2024-09-10T22:35:44.661786+0000","last_undegraded":"2024-09-10T22:35:44.661786+0000","last_fullsized":"2024-09-10T22:35:44.661786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:17.176523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.653565+0000","last_change":"2024-09-10T22:35:43.655647+0000","last_active":"2024-09-10T22:35:45.653565+0000","last_peered":"2024-09-10T22:35:45.653565+0000","last_clean":"2024-09-10T22:35:45.653565+0000","last_became_active":"2024-09-10T22:35:43.655346+0000","last_became_peered":"2024-09-10T22:35:43.655346+0000","last_unstale":"2024-09-10T22:35:45.653565+0000","last_undegraded":"2024-09-10T22:35:45.653565+0000","last_fullsized":"2024-09-10T22:35:45.653565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:16.171929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.665121+0000","last_change":"2024-09-10T22:35:44.662292+0000","last_active":"2024-09-10T22:35:44.665121+0000","last_peered":"2024-09-10T22:35:44.665121+0000","last_clean":"2024-09-10T22:35:44.665121+0000","last_became_active":"2024-09-10T22:35:44.662032+0000","last_became_peered":"2024-09-10T22:35:44.662032+0000","last_unstale":"2024-09-10T22:35:44.665121+0000","last_undegraded":"2024-09-10T22:35:44.665121+0000","last_fullsized":"2024-09-10T22:35:44.665121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:31.822145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.672112+0000","last_change":"2024-09-10T22:35:43.656104+0000","last_active":"2024-09-10T22:35:44.672112+0000","last_peered":"2024-09-10T22:35:44.672112+0000","last_clean":"2024-09-10T22:35:44.672112+0000","last_became_active":"2024-09-10T22:35:43.655798+0000","last_became_peered":"2024-09-10T22:35:43.655798+0000","last_unstale":"2024-09-10T22:35:44.672112+0000","last_undegraded":"2024-09-10T22:35:44.672112+0000","last_fullsized":"2024-09-10T22:35:44.672112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:48.438149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.655033+0000","last_change":"2024-09-10T22:35:44.669098+0000","last_active":"2024-09-10T22:35:45.655033+0000","last_peered":"2024-09-10T22:35:45.655033+0000","last_clean":"2024-09-10T22:35:45.655033+0000","last_became_active":"2024-09-10T22:35:44.668486+0000","last_became_peered":"2024-09-10T22:35:44.668486+0000","last_unstale":"2024-09-10T22:35:45.655033+0000","last_undegraded":"2024-09-10T22:35:45.655033+0000","last_fullsized":"2024-09-10T22:35:45.655033+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:01.236708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654728+0000","last_change":"2024-09-10T22:35:43.659628+0000","last_active":"2024-09-10T22:35:45.654728+0000","last_peered":"2024-09-10T22:35:45.654728+0000","last_clean":"2024-09-10T22:35:45.654728+0000","last_became_active":"2024-09-10T22:35:43.659308+0000","last_became_peered":"2024-09-10T22:35:43.659308+0000","last_unstale":"2024-09-10T22:35:45.654728+0000","last_undegraded":"2024-09-10T22:35:45.654728+0000","last_fullsized":"2024-09-10T22:35:45.654728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:08:22.440344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652896+0000","last_change":"2024-09-10T22:35:44.657571+0000","last_active":"2024-09-10T22:35:45.652896+0000","last_peered":"2024-09-10T22:35:45.652896+0000","last_clean":"2024-09-10T22:35:45.652896+0000","last_became_active":"2024-09-10T22:35:44.657300+0000","last_became_peered":"2024-09-10T22:35:44.657300+0000","last_unstale":"2024-09-10T22:35:45.652896+0000","last_undegraded":"2024-09-10T22:35:45.652896+0000","last_fullsized":"2024-09-10T22:35:45.652896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:54.836977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.650837+0000","last_change":"2024-09-10T22:35:43.655029+0000","last_active":"2024-09-10T22:35:44.650837+0000","last_peered":"2024-09-10T22:35:44.650837+0000","last_clean":"2024-09-10T22:35:44.650837+0000","last_became_active":"2024-09-10T22:35:43.654732+0000","last_became_peered":"2024-09-10T22:35:43.654732+0000","last_unstale":"2024-09-10T22:35:44.650837+0000","last_undegraded":"2024-09-10T22:35:44.650837+0000","last_fullsized":"2024-09-10T22:35:44.650837+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:45.095939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.657959+0000","last_change":"2024-09-10T22:35:44.655653+0000","last_active":"2024-09-10T22:35:44.657959+0000","last_peered":"2024-09-10T22:35:44.657959+0000","last_clean":"2024-09-10T22:35:44.657959+0000","last_became_active":"2024-09-10T22:35:44.655373+0000","last_became_peered":"2024-09-10T22:35:44.655373+0000","last_unstale":"2024-09-10T22:35:44.657959+0000","last_undegraded":"2024-09-10T22:35:44.657959+0000","last_fullsized":"2024-09-10T22:35:44.657959+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:02.446422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652896+0000","last_change":"2024-09-10T22:35:43.653478+0000","last_active":"2024-09-10T22:35:45.652896+0000","last_peered":"2024-09-10T22:35:45.652896+0000","last_clean":"2024-09-10T22:35:45.652896+0000","last_became_active":"2024-09-10T22:35:43.653135+0000","last_became_peered":"2024-09-10T22:35:43.653135+0000","last_unstale":"2024-09-10T22:35:45.652896+0000","last_undegraded":"2024-09-10T22:35:45.652896+0000","last_fullsized":"2024-09-10T22:35:45.652896+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:51.714228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.652782+0000","last_change":"2024-09-10T22:35:44.655629+0000","last_active":"2024-09-10T22:35:45.652782+0000","last_peered":"2024-09-10T22:35:45.652782+0000","last_clean":"2024-09-10T22:35:45.652782+0000","last_became_active":"2024-09-10T22:35:44.654546+0000","last_became_peered":"2024-09-10T22:35:44.654546+0000","last_unstale":"2024-09-10T22:35:45.652782+0000","last_undegraded":"2024-09-10T22:35:45.652782+0000","last_fullsized":"2024-09-10T22:35:45.652782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:25.975720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651552+0000","last_change":"2024-09-10T22:35:43.655195+0000","last_active":"2024-09-10T22:35:44.651552+0000","last_peered":"2024-09-10T22:35:44.651552+0000","last_clean":"2024-09-10T22:35:44.651552+0000","last_became_active":"2024-09-10T22:35:43.654881+0000","last_became_peered":"2024-09-10T22:35:43.654881+0000","last_unstale":"2024-09-10T22:35:44.651552+0000","last_undegraded":"2024-09-10T22:35:44.651552+0000","last_fullsized":"2024-09-10T22:35:44.651552+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:10.021023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.675054+0000","last_change":"2024-09-10T22:35:44.672917+0000","last_active":"2024-09-10T22:35:44.675054+0000","last_peered":"2024-09-10T22:35:44.675054+0000","last_clean":"2024-09-10T22:35:44.675054+0000","last_became_active":"2024-09-10T22:35:44.672644+0000","last_became_peered":"2024-09-10T22:35:44.672644+0000","last_unstale":"2024-09-10T22:35:44.675054+0000","last_undegraded":"2024-09-10T22:35:44.675054+0000","last_fullsized":"2024-09-10T22:35:44.675054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:37.104143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654238+0000","last_change":"2024-09-10T22:35:43.652113+0000","last_active":"2024-09-10T22:35:45.654238+0000","last_peered":"2024-09-10T22:35:45.654238+0000","last_clean":"2024-09-10T22:35:45.654238+0000","last_became_active":"2024-09-10T22:35:43.649256+0000","last_became_peered":"2024-09-10T22:35:43.649256+0000","last_unstale":"2024-09-10T22:35:45.654238+0000","last_undegraded":"2024-09-10T22:35:45.654238+0000","last_fullsized":"2024-09-10T22:35:45.654238+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:01.441828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T22:35:45.654154+0000","last_change":"2024-09-10T22:35:44.665221+0000","last_active":"2024-09-10T22:35:45.654154+0000","last_peered":"2024-09-10T22:35:45.654154+0000","last_clean":"2024-09-10T22:35:45.654154+0000","last_became_active":"2024-09-10T22:35:44.664952+0000","last_became_peered":"2024-09-10T22:35:44.664952+0000","last_unstale":"2024-09-10T22:35:45.654154+0000","last_undegraded":"2024-09-10T22:35:45.654154+0000","last_fullsized":"2024-09-10T22:35:45.654154+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:54.836977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.652314+0000","last_change":"2024-09-10T22:35:43.660238+0000","last_active":"2024-09-10T22:35:44.652314+0000","last_peered":"2024-09-10T22:35:44.652314+0000","last_clean":"2024-09-10T22:35:44.652314+0000","last_became_active":"2024-09-10T22:35:43.659954+0000","last_became_peered":"2024-09-10T22:35:43.659954+0000","last_unstale":"2024-09-10T22:35:44.652314+0000","last_undegraded":"2024-09-10T22:35:44.652314+0000","last_fullsized":"2024-09-10T22:35:44.652314+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:20.614603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.663424+0000","last_change":"2024-09-10T22:35:44.660556+0000","last_active":"2024-09-10T22:35:44.663424+0000","last_peered":"2024-09-10T22:35:44.663424+0000","last_clean":"2024-09-10T22:35:44.663424+0000","last_became_active":"2024-09-10T22:35:44.660289+0000","last_became_peered":"2024-09-10T22:35:44.660289+0000","last_unstale":"2024-09-10T22:35:44.663424+0000","last_undegraded":"2024-09-10T22:35:44.663424+0000","last_fullsized":"2024-09-10T22:35:44.663424+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:43.634328+0000","last_clean_scrub_stamp":"2024-09-10T22:35:43.634328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:33.230297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.647759+0000","last_change":"2024-09-10T22:35:43.654344+0000","last_active":"2024-09-10T22:35:44.647759+0000","last_peered":"2024-09-10T22:35:44.647759+0000","last_clean":"2024-09-10T22:35:44.647759+0000","last_became_active":"2024-09-10T22:35:43.654071+0000","last_became_peered":"2024-09-10T22:35:43.654071+0000","last_unstale":"2024-09-10T22:35:44.647759+0000","last_undegraded":"2024-09-10T22:35:44.647759+0000","last_fullsized":"2024-09-10T22:35:44.647759+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:35:42.629733+0000","last_clean_scrub_stamp":"2024-09-10T22:35:42.629733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:02.734433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":128,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T22:35:44.651702+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:35:44.651702+0000","last_peered":"2024-09-10T22:35:44.651702+0000","last_clean":"2024-09-10T22:35:44.651702+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:35:44.651702+0000","last_undegraded":"2024-09-10T22:35:44.651702+0000","last_fullsized":"2024-09-10T22:35:44.651702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":937984,"data_stored":719671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":348160,"data_stored":129829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":360448,"data_stored":136185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":360448,"data_stored":135617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":725881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":356352,"data_stored":130847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":344064,"data_stored":129283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":356352,"data_stored":130847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:35:49.019 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:35:49.019 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:35:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 -- 172.21.15.66:0/946715463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 msgr2=0x7f88800af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 --2- 172.21.15.66:0/946715463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800af900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8880073780 tx=0x7f887802aac0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 -- 172.21.15.66:0/946715463 shutdown_connections 2024-09-10T22:35:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 --2- 172.21.15.66:0/946715463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800af900 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 --2- 172.21.15.66:0/946715463 >> v2:172.21.15.94:3300/0 conn(0x7f88800b2d70 0x7f88800b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 -- 172.21.15.66:0/946715463 >> 172.21.15.66:0/946715463 conn(0x7f88800d1db0 msgr2=0x7f88800d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 -- 172.21.15.66:0/946715463 shutdown_connections 2024-09-10T22:35:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.167+0000 7f888874b640 1 -- 172.21.15.66:0/946715463 wait complete. 2024-09-10T22:35:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.168+0000 7f888874b640 1 Processor -- start 2024-09-10T22:35:49.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.168+0000 7f888874b640 1 -- start start 2024-09-10T22:35:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f888874b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f888874b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88800b2d70 0x7f88800cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f888874b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f888015f5b0 0x7f88800b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f888874b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f88800b33d0 con 0x7f88800af490 2024-09-10T22:35:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f888874b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88800b2310 con 0x7f888015f5b0 2024-09-10T22:35:49.172 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f888874b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f88800afe40 con 0x7f88800b2d70 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52434/0 (socket says 172.21.15.66:52434) 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 -- 172.21.15.66:0/4157366023 learned_addr learned my addr 172.21.15.66:0/4157366023 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f8885ce6640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88800b2d70 0x7f88800cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f8886ce8640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f888015f5b0 0x7f88800b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 -- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88800b2d70 msgr2=0x7f88800cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88800b2d70 0x7f88800cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 -- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f888015f5b0 msgr2=0x7f88800b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f888015f5b0 0x7f88800b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 -- 172.21.15.66:0/4157366023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88800b6860 con 0x7f88800af490 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.169+0000 7f88864e7640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800cb790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c3a0 tx=0x7f887c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.170+0000 7f8876ffd640 1 -- 172.21.15.66:0/4157366023 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c017070 con 0x7f88800af490 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.170+0000 7f8876ffd640 1 -- 172.21.15.66:0/4157366023 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f887c0053c0 con 0x7f88800af490 2024-09-10T22:35:49.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.170+0000 7f8876ffd640 1 -- 172.21.15.66:0/4157366023 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c003850 con 0x7f88800af490 2024-09-10T22:35:49.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.170+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88800b6b40 con 0x7f88800af490 2024-09-10T22:35:49.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.170+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88801f5da0 con 0x7f88800af490 2024-09-10T22:35:49.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.171+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88800af900 con 0x7f88800af490 2024-09-10T22:35:49.174 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.171+0000 7f8876ffd640 1 -- 172.21.15.66:0/4157366023 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f887c0039f0 con 0x7f88800af490 2024-09-10T22:35:49.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.172+0000 7f8876ffd640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8864077f70 0x7f886407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.172+0000 7f8876ffd640 1 -- 172.21.15.66:0/4157366023 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f887c099300 con 0x7f88800af490 2024-09-10T22:35:49.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.172+0000 7f8885ce6640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8864077f70 0x7f886407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.175 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.172+0000 7f8885ce6640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8864077f70 0x7f886407a430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88800cef20 tx=0x7f8870005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:49.178 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.176+0000 7f8876ffd640 1 -- 172.21.15.66:0/4157366023 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f887c0617a0 con 0x7f88800af490 2024-09-10T22:35:49.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.438+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f888015b170 con 0x7f88800af490 2024-09-10T22:35:49.442 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.440+0000 7f8876ffd640 1 -- 172.21.15.66:0/4157366023 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f887c0661d0 con 0x7f88800af490 2024-09-10T22:35:49.442 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:35:49.442 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmyY04GBAAcxW03/29zpbpxXzE+JrsFA== 2024-09-10T22:35:49.442 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:35:49.442 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:35:49.442 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:35:49.443 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:35:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8864077f70 msgr2=0x7f886407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8864077f70 0x7f886407a430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88800cef20 tx=0x7f8870005f90 comp rx=0 tx=0).stop 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 msgr2=0x7f88800cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800cb790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c3a0 tx=0x7f887c00c860 comp rx=0 tx=0).stop 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 shutdown_connections 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8864077f70 0x7f886407a430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f888015f5b0 0x7f88800b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f88800b2d70 0x7f88800cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 --2- 172.21.15.66:0/4157366023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88800af490 0x7f88800cb790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 >> 172.21.15.66:0/4157366023 conn(0x7f88800d1db0 msgr2=0x7f888015c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 shutdown_connections 2024-09-10T22:35:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.443+0000 7f888874b640 1 -- 172.21.15.66:0/4157366023 wait complete. 2024-09-10T22:35:49.456 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.559+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/3766245374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5be0003a00 msgr2=0x7f5be0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.559+0000 7f5bfb2ed640 1 --2- 172.21.15.66:0/3766245374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5be0003a00 0x7f5be0003e20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009b50 tx=0x7f5be0005830 comp rx=0 tx=0).stop 2024-09-10T22:35:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.559+0000 7f5bf988a640 1 -- 172.21.15.66:0/3766245374 reap_dead start 2024-09-10T22:35:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.559+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/3766245374 shutdown_connections 2024-09-10T22:35:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.559+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/3766245374 >> 172.21.15.66:0/3766245374 conn(0x7f5bf40d0c90 msgr2=0x7f5bf40d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.559+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/3766245374 shutdown_connections 2024-09-10T22:35:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.559+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/3766245374 wait complete. 2024-09-10T22:35:49.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.561+0000 7f5bfb2ed640 1 Processor -- start 2024-09-10T22:35:49.563 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.561+0000 7f5bfb2ed640 1 -- start start 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.561+0000 7f5bfb2ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5be0003a00 0x7f5bf40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bfb2ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf41514e0 0x7f5bf40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bfb2ed640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bf40cfb40 0x7f5bf40af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bfb2ed640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40cb4a0 con 0x7f5bf41514e0 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bfb2ed640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40cb320 con 0x7f5be0003a00 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bfb2ed640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf40cb620 con 0x7f5bf40cfb40 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf41514e0 0x7f5bf40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.565 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf41514e0 0x7f5bf40aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52450/0 (socket says 172.21.15.66:52450) 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 -- 172.21.15.66:0/1273396392 learned_addr learned my addr 172.21.15.66:0/1273396392 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf988a640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bf40cfb40 0x7f5bf40af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf9089640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5be0003a00 0x7f5bf40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 -- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bf40cfb40 msgr2=0x7f5bf40af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bf40cfb40 0x7f5bf40af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 -- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5be0003a00 msgr2=0x7f5bf40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5be0003a00 0x7f5bf40cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 -- 172.21.15.66:0/1273396392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf40d00c0 con 0x7f5bf41514e0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bf8888640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf41514e0 0x7f5bf40aeda0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5be4001050 tx=0x7f5be40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5be9ffb640 1 -- 172.21.15.66:0/1273396392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4015070 con 0x7f5bf41514e0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5be9ffb640 1 -- 172.21.15.66:0/1273396392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be4011860 con 0x7f5bf41514e0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5be9ffb640 1 -- 172.21.15.66:0/1273396392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4011cb0 con 0x7f5bf41514e0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.562+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf40d03a0 con 0x7f5bf41514e0 2024-09-10T22:35:49.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.563+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf40c95a0 con 0x7f5bf41514e0 2024-09-10T22:35:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.564+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf406cb20 con 0x7f5bf41514e0 2024-09-10T22:35:49.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.564+0000 7f5be9ffb640 1 -- 172.21.15.66:0/1273396392 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5be40103c0 con 0x7f5bf41514e0 2024-09-10T22:35:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.564+0000 7f5be9ffb640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5bd8077fa0 0x7f5bd807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.565+0000 7f5be9ffb640 1 -- 172.21.15.66:0/1273396392 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5be409aca0 con 0x7f5bf41514e0 2024-09-10T22:35:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.565+0000 7f5bf9089640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5bd8077fa0 0x7f5bd807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:49.568 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.565+0000 7f5bf9089640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5bd8077fa0 0x7f5bd807a460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0035000 tx=0x7f5bf002d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:49.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.569+0000 7f5be9ffb640 1 -- 172.21.15.66:0/1273396392 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5bf406d670 con 0x7f5bf41514e0 2024-09-10T22:35:49.627 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:35:49.625+0000 7fb082a4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:35:49.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.791+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bf415f570 con 0x7f5bf41514e0 2024-09-10T22:35:49.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.791+0000 7f5be9ffb640 1 -- 172.21.15.66:0/1273396392 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f5be4063150 con 0x7f5bf41514e0 2024-09-10T22:35:49.793 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:49.794 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"btime":"2024-09-10T22:35:47:657322+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/307839984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":307839984},{"type":"v1","addr":"172.21.15.94:6836","nonce":307839984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2042084416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2042084416},{"type":"v1","addr":"172.21.15.94:6837","nonce":2042084416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1521655589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1521655589},{"type":"v1","addr":"172.21.15.66:6834","nonce":1521655589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:47.159794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4846},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4846":{"gid":4846,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1970715211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1970715211},{"type":"v1","addr":"172.21.15.66:6835","nonce":1970715211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4846,"qdb_cluster":[4846]},"id":2}]} 2024-09-10T22:35:49.794 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-09-10T22:35:49.795 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5bd8077fa0 msgr2=0x7f5bd807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5bd8077fa0 0x7f5bd807a460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0035000 tx=0x7f5bf002d040 comp rx=0 tx=0).stop 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf41514e0 msgr2=0x7f5bf40aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf41514e0 0x7f5bf40aeda0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5be4001050 tx=0x7f5be40039c0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 shutdown_connections 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5bd8077fa0 0x7f5bd807a460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5bf40cfb40 0x7f5bf40af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bf41514e0 0x7f5bf40aeda0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 --2- 172.21.15.66:0/1273396392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5be0003a00 0x7f5bf40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 >> 172.21.15.66:0/1273396392 conn(0x7f5bf40d0c90 msgr2=0x7f5bf40b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 shutdown_connections 2024-09-10T22:35:49.796 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:49.793+0000 7f5bfb2ed640 1 -- 172.21.15.66:0/1273396392 wait complete. 2024-09-10T22:35:50.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:35:44.016061+0000', 'modified': '2024-09-10T22:35:47.159794+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4846}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4846': {'gid': 4846, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/1970715211', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 1970715211}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 1970715211}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4846, 'qdb_cluster': [4846]} 2024-09-10T22:35:50.209 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:35:50.209 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 -- 172.21.15.66:0/323486356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3015dfd0 msgr2=0x7efd300738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 --2- 172.21.15.66:0/323486356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3015dfd0 0x7efd300738a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005fd0 tx=0x7efd2c02a5d0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 -- 172.21.15.66:0/323486356 shutdown_connections 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 --2- 172.21.15.66:0/323486356 >> v2:172.21.15.94:3300/0 conn(0x7efd30070d70 0x7efd3006cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 --2- 172.21.15.66:0/323486356 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3015dfd0 0x7efd300738a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 -- 172.21.15.66:0/323486356 >> 172.21.15.66:0/323486356 conn(0x7efd300b2750 msgr2=0x7efd300b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 -- 172.21.15.66:0/323486356 shutdown_connections 2024-09-10T22:35:50.320 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.317+0000 7efd37edd640 1 -- 172.21.15.66:0/323486356 wait complete. 2024-09-10T22:35:50.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 Processor -- start 2024-09-10T22:35:50.321 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 -- start start 2024-09-10T22:35:50.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3006d060 0x7efd300cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd30070d70 0x7efd3014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3015dfd0 0x7efd3014ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd3006fd30 con 0x7efd30070d70 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd300af9c0 con 0x7efd3015dfd0 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.319+0000 7efd37edd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efd3006feb0 con 0x7efd3006d060 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd30070d70 0x7efd3014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd30070d70 0x7efd3014e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52464/0 (socket says 172.21.15.66:52464) 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 -- 172.21.15.66:0/3211501449 learned_addr learned my addr 172.21.15.66:0/3211501449 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35c79640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3006d060 0x7efd300cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd3647a640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3015dfd0 0x7efd3014ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 -- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3006d060 msgr2=0x7efd300cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3006d060 0x7efd300cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 -- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3015dfd0 msgr2=0x7efd3014ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.328 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3015dfd0 0x7efd3014ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 -- 172.21.15.66:0/3211501449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3014f180 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd35478640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd30070d70 0x7efd3014e700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd2000e9c0 tx=0x7efd2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd267fc640 1 -- 172.21.15.66:0/3211501449 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2000cd80 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd267fc640 1 -- 172.21.15.66:0/3211501449 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd20004020 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3014f430 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd267fc640 1 -- 172.21.15.66:0/3211501449 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20004450 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.320+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd300cd680 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.322+0000 7efd267fc640 1 -- 172.21.15.66:0/3211501449 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efd200035f0 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.323+0000 7efd267fc640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efd14077f70 0x7efd1407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.323+0000 7efd267fc640 1 -- 172.21.15.66:0/3211501449 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7efd2009a6d0 con 0x7efd30070d70 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.323+0000 7efd35c79640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efd14077f70 0x7efd1407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.323+0000 7efd35c79640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efd14077f70 0x7efd1407a430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd2c033000 tx=0x7efd2c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.329 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.323+0000 7efd3acac740 1 Processor -- start 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd3acac740 1 -- start start 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd3acac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50840b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5083e5f0 0x55cb5083ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd3acac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb507b5820 con 0x55cb508691b0 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb507a4c50 con 0x55cb50888df0 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55cb507a8a60 con 0x55cb5083e5f0 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd35c79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50840b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd35c79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50840b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52468/0 (socket says 172.21.15.66:52468) 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.324+0000 7efd35c79640 1 -- 172.21.15.66:0/1992095040 learned_addr learned my addr 172.21.15.66:0/1992095040 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35478640 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd3647a640 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5083e5f0 0x55cb5083ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35c79640 1 -- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5083e5f0 msgr2=0x55cb5083ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35c79640 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5083e5f0 0x55cb5083ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35c79640 1 -- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 msgr2=0x55cb508891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35c79640 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35c79640 1 -- 172.21.15.66:0/1992095040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb50506850 con 0x55cb508691b0 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35478640 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd35c79640 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50840b20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006010 tx=0x7efd2c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4177bcaf322b6156 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd137fe640 1 -- 172.21.15.66:0/1992095040 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c035040 con 0x55cb508691b0 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd137fe640 1 -- 172.21.15.66:0/1992095040 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd2c04d070 con 0x55cb508691b0 2024-09-10T22:35:50.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.325+0000 7efd137fe640 1 -- 172.21.15.66:0/1992095040 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c03c6d0 con 0x55cb508691b0 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 -- 172.21.15.66:0/1992095040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 msgr2=0x55cb50840b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50840b20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006010 tx=0x7efd2c03c000 comp rx=0 tx=0).stop 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 -- 172.21.15.66:0/1992095040 shutdown_connections 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5083e5f0 0x55cb5083ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 --2- 172.21.15.66:0/1992095040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50840b20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 -- 172.21.15.66:0/1992095040 >> 172.21.15.66:0/1992095040 conn(0x55cb5083f6a0 msgr2=0x55cb508430c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 -- 172.21.15.66:0/1992095040 shutdown_connections 2024-09-10T22:35:50.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 -- 172.21.15.66:0/1992095040 wait complete. 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.326+0000 7efd3acac740 1 -- start start 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.327+0000 7efd3acac740 1 Processor -- start 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.327+0000 7efd3acac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.327+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50855dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.327+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50864be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.327+0000 7efd3acac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb50515620 con 0x55cb508691b0 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.327+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb5074a5f0 con 0x55cb50888df0 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.327+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55cb5083d890 con 0x55cb5087fae0 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd3acac740 10 client.0 _ll_register_callbacks cb 0x55cb5074e0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52480/0 (socket says 172.21.15.66:52480) 2024-09-10T22:35:50.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 -- 172.21.15.66:0/591656746 learned_addr learned my addr 172.21.15.66:0/591656746 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35478640 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50855dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd3647a640 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50864be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 -- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 msgr2=0x55cb50864be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50864be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 -- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 msgr2=0x55cb50855dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50855dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 -- 172.21.15.66:0/591656746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5057d6b0 con 0x55cb508691b0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd35c79640 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861550 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efd2c02abc0 tx=0x7efd2c03c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b5d70c359e539e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd10ff9640 1 -- 172.21.15.66:0/591656746 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c057020 con 0x55cb508691b0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd10ff9640 1 -- 172.21.15.66:0/591656746 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7efd2c003ef0 con 0x55cb508691b0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c04d070 con 0x55cb508691b0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd10ff9640 1 -- 172.21.15.66:0/591656746 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7efd2c061050 con 0x55cb508691b0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd3acac740 10 client.5073 fetch_fsmap learned FSMap version 17 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.328+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55cb508807a0 con 0x55cb508691b0 2024-09-10T22:35:50.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd10ff9640 1 -- 172.21.15.66:0/591656746 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd2c04d070 con 0x55cb508691b0 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd10ff9640 1 -- 172.21.15.66:0/591656746 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7efd2c064020 con 0x55cb508691b0 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd3acac740 10 client.5073 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd3acac740 10 client.5073 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd3acac740 20 client.5073 populate_metadata read hostname 'smithi066' 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd3acac740 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50881400 0x55cb5086b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd3acac740 4 client.5073 mds_command: new command op to 4846 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 --> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb507b7770 con 0x55cb50881400 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd35478640 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50881400 0x55cb5086b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd10ff9640 10 client.5073 ms_handle_connect on v2:172.21.15.66:6833/1970715211 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.329+0000 7efd35478640 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50881400 0x55cb5086b1c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a601ea54f7f58e server_cookie=7e197a387cea076c in_seq=0 out_seq=0 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd10ff9640 1 -- 172.21.15.66:0/591656746 <== mds.0 v2:172.21.15.66:6833/1970715211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb507b7770 con 0x55cb50881400 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd10ff9640 10 client.5073 handle_command_reply: tid=0 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 client.5073 shutdown 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 msgr2=0x55cb50861550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.339 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861550 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efd2c02abc0 tx=0x7efd2c03c840 comp rx=0 tx=0).stop 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 shutdown_connections 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50881400 0x55cb5086b1c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50864be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50855dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 --2- 172.21.15.66:0/591656746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861550 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd10ff9640 0 client.5073 ms_handle_reset on v2:172.21.15.66:6833/1970715211 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 >> 172.21.15.66:0/591656746 conn(0x55cb5083e5f0 msgr2=0x55cb50869590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 shutdown_connections 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 1 -- 172.21.15.66:0/591656746 wait complete. 2024-09-10T22:35:50.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.330+0000 7efd3acac740 20 client.5073 trim_cache size 0 max 16384 2024-09-10T22:35:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 Processor -- start 2024-09-10T22:35:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 -- start start 2024-09-10T22:35:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50861a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50888220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efd300b50f0 con 0x55cb508691b0 2024-09-10T22:35:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd301f84a0 con 0x55cb50888df0 2024-09-10T22:35:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efd300b4aa0 con 0x55cb5087fae0 2024-09-10T22:35:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd35c79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd35c79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52494/0 (socket says 172.21.15.66:52494) 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.349+0000 7efd35c79640 1 -- 172.21.15.66:0/836975398 learned_addr learned my addr 172.21.15.66:0/836975398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35478640 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50861a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd3647a640 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50888220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35c79640 1 -- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 msgr2=0x55cb50861a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35c79640 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50861a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35c79640 1 -- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 msgr2=0x55cb50888220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35c79640 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50888220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35c79640 1 -- 172.21.15.66:0/836975398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb50897880 con 0x55cb508691b0 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35478640 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50861a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd3647a640 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50888220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd35c79640 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009230 tx=0x7efd2c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b86ae330051f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd12ffd640 1 -- 172.21.15.66:0/836975398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c057040 con 0x55cb508691b0 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd12ffd640 1 -- 172.21.15.66:0/836975398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd2c04d040 con 0x55cb508691b0 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd12ffd640 1 -- 172.21.15.66:0/836975398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c049470 con 0x55cb508691b0 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd3acac740 1 -- 172.21.15.66:0/836975398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 msgr2=0x55cb50861530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.350+0000 7efd3acac740 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861530 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009230 tx=0x7efd2c009620 comp rx=0 tx=0).stop 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 -- 172.21.15.66:0/836975398 shutdown_connections 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb50888220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb5087fae0 0x55cb50861a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 --2- 172.21.15.66:0/836975398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb508691b0 0x55cb50861530 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 -- 172.21.15.66:0/836975398 >> 172.21.15.66:0/836975398 conn(0x55cb5083e5f0 msgr2=0x55cb508649f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 -- 172.21.15.66:0/836975398 shutdown_connections 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 -- 172.21.15.66:0/836975398 wait complete. 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 -- start start 2024-09-10T22:35:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.351+0000 7efd3acac740 1 Processor -- start 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb508691b0 0x55cb5086b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3acac740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb5087fae0 0x55cb508f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3acac740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3acac740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55cb50881280 con 0x55cb5087fae0 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55cb50880a80 con 0x55cb50888df0 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3acac740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55cb508818d0 con 0x55cb508691b0 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3acac740 10 client.0 _ll_register_callbacks cb 0x7efd200046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:35:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb5087fae0 0x55cb508f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb5087fae0 0x55cb508f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52496/0 (socket says 172.21.15.66:52496) 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 -- 172.21.15.66:0/1544154455 learned_addr learned my addr 172.21.15.66:0/1544154455 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35c79640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb508691b0 0x55cb5086b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3647a640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 -- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb508691b0 msgr2=0x55cb5086b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb508691b0 0x55cb5086b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 -- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 msgr2=0x55cb508ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd35478640 1 -- 172.21.15.66:0/1544154455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5087e0b0 con 0x55cb5087fae0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.352+0000 7efd3647a640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd35478640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb5087fae0 0x55cb508f1fc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efd20086010 tx=0x7efd20009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf13f21e90acfd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd10ff9640 1 -- 172.21.15.66:0/1544154455 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20097070 con 0x55cb5087fae0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd10ff9640 1 -- 172.21.15.66:0/1544154455 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7efd20067320 con 0x55cb5087fae0 2024-09-10T22:35:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c04d040 con 0x55cb5087fae0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd10ff9640 1 -- 172.21.15.66:0/1544154455 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7efd20062b80 con 0x55cb5087fae0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 10 client.5079 fetch_fsmap learned FSMap version 17 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55cb508f2500 con 0x55cb5087fae0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd10ff9640 1 -- 172.21.15.66:0/1544154455 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd2c04d040 con 0x55cb5087fae0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd10ff9640 1 -- 172.21.15.66:0/1544154455 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7efd2006bb40 con 0x55cb5087fae0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 10 client.5079 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 10 client.5079 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 20 client.5079 populate_metadata read hostname 'smithi066' 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50854cf0 0x55cb50867710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 4 client.5079 mds_command: new command op to 4846 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 --> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] -- command(tid 0: {"prefix": "status"}) -- 0x55cb507b3220 con 0x55cb50854cf0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.353+0000 7efd35c79640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50854cf0 0x55cb50867710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd35c79640 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50854cf0 0x55cb50867710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a6a532d2b63939 server_cookie=e12d4e5dc63638a3 in_seq=0 out_seq=0 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd10ff9640 10 client.5079 ms_handle_connect on v2:172.21.15.66:6833/1970715211 2024-09-10T22:35:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd10ff9640 1 -- 172.21.15.66:0/1544154455 <== mds.0 v2:172.21.15.66:6833/1970715211 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55cb507b3220 con 0x55cb50854cf0 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd10ff9640 10 client.5079 handle_command_reply: tid=0 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 client.5079 shutdown 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb5087fae0 msgr2=0x55cb508f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb5087fae0 0x55cb508f1fc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efd20086010 tx=0x7efd20009780 comp rx=0 tx=0).stop 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 shutdown_connections 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55cb50854cf0 0x55cb50867710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb50888df0 0x55cb508ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd10ff9640 0 client.5079 ms_handle_reset on v2:172.21.15.66:6833/1970715211 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cb5087fae0 0x55cb508f1fc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 --2- 172.21.15.66:0/1544154455 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55cb508691b0 0x55cb5086b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 >> 172.21.15.66:0/1544154455 conn(0x55cb5083e5f0 msgr2=0x55cb50889260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 shutdown_connections 2024-09-10T22:35:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 1 -- 172.21.15.66:0/1544154455 wait complete. 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.354+0000 7efd3acac740 20 client.5079 trim_cache size 0 max 16384 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "7ef137cd-a1ce-4819-b27d-b71268612f85", 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4846, 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 5.7082142009999997, 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 17, 2024-09-10T22:35:50.363 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 32, 2024-09-10T22:35:50.364 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 32, 2024-09-10T22:35:50.364 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 12.590341316 2024-09-10T22:35:50.364 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:35:50.364 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efd14077f70 msgr2=0x7efd1407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efd14077f70 0x7efd1407a430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd2c033000 tx=0x7efd2c004730 comp rx=0 tx=0).stop 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd30070d70 msgr2=0x7efd3014e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd30070d70 0x7efd3014e700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd2000e9c0 tx=0x7efd2000ee80 comp rx=0 tx=0).stop 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 shutdown_connections 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efd14077f70 0x7efd1407a430 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd3015dfd0 0x7efd3014ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd30070d70 0x7efd3014e700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 --2- 172.21.15.66:0/3211501449 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efd3006d060 0x7efd300cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 >> 172.21.15.66:0/3211501449 conn(0x7efd300b2750 msgr2=0x7efd30161070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 shutdown_connections 2024-09-10T22:35:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.355+0000 7efd37edd640 1 -- 172.21.15.66:0/3211501449 wait complete. 2024-09-10T22:35:50.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:35:50.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.471+0000 7fe61973d640 1 -- 172.21.15.66:0/3625586302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d2be0 msgr2=0x7fe6140d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.471+0000 7fe61973d640 1 --2- 172.21.15.66:0/3625586302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d2be0 0x7fe6140d1180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009830 tx=0x7fe5fc02a5c0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.471+0000 7fe61973d640 1 -- 172.21.15.66:0/3625586302 shutdown_connections 2024-09-10T22:35:50.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.471+0000 7fe61973d640 1 --2- 172.21.15.66:0/3625586302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d2be0 0x7fe6140d1180 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.471+0000 7fe61973d640 1 -- 172.21.15.66:0/3625586302 >> 172.21.15.66:0/3625586302 conn(0x7fe6140d3f10 msgr2=0x7fe6140d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.471+0000 7fe61973d640 1 -- 172.21.15.66:0/3625586302 shutdown_connections 2024-09-10T22:35:50.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.471+0000 7fe61973d640 1 -- 172.21.15.66:0/3625586302 wait complete. 2024-09-10T22:35:50.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 Processor -- start 2024-09-10T22:35:50.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 -- start start 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d1ae0 0x7fe6140cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6140d2be0 0x7fe614151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61415e090 0x7fe61415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe6140b4ef0 con 0x7fe6140d1ae0 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6140b4d70 con 0x7fe6140d2be0 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.473+0000 7fe61973d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe6140b5070 con 0x7fe61415e090 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d1ae0 0x7fe6140cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d1ae0 0x7fe6140cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52512/0 (socket says 172.21.15.66:52512) 2024-09-10T22:35:50.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 -- 172.21.15.66:0/623059160 learned_addr learned my addr 172.21.15.66:0/623059160 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe6137fe640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61415e090 0x7fe61415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe6127fc640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6140d2be0 0x7fe614151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 -- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61415e090 msgr2=0x7fe61415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61415e090 0x7fe61415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 -- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6140d2be0 msgr2=0x7fe614151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6140d2be0 0x7fe614151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 -- 172.21.15.66:0/623059160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6140cc660 con 0x7fe6140d1ae0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe612ffd640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d1ae0 0x7fe6140cd6e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00c730 tx=0x7fe60c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe60b7fe640 1 -- 172.21.15.66:0/623059160 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c003b70 con 0x7fe6140d1ae0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe60b7fe640 1 -- 172.21.15.66:0/623059160 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe60c003d30 con 0x7fe6140d1ae0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6140cc940 con 0x7fe6140d1ae0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe60b7fe640 1 -- 172.21.15.66:0/623059160 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00aab0 con 0x7fe6140d1ae0 2024-09-10T22:35:50.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.474+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6140ccf00 con 0x7fe6140d1ae0 2024-09-10T22:35:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.475+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6140b5520 con 0x7fe6140d1ae0 2024-09-10T22:35:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.476+0000 7fe60b7fe640 1 -- 172.21.15.66:0/623059160 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe60c00ac50 con 0x7fe6140d1ae0 2024-09-10T22:35:50.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.477+0000 7fe60b7fe640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe5f4077f70 0x7fe5f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:50.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.477+0000 7fe60b7fe640 1 -- 172.21.15.66:0/623059160 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe60c0999f0 con 0x7fe6140d1ae0 2024-09-10T22:35:50.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.477+0000 7fe6127fc640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe5f4077f70 0x7fe5f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:50.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.477+0000 7fe6127fc640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe5f4077f70 0x7fe5f407a430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc034000 tx=0x7fe5fc004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:50.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.481+0000 7fe60b7fe640 1 -- 172.21.15.66:0/623059160 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe60c061ea0 con 0x7fe6140d1ae0 2024-09-10T22:35:50.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.706+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe61406d650 con 0x7fe6140d1ae0 2024-09-10T22:35:50.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.706+0000 7fe60b7fe640 1 -- 172.21.15.66:0/623059160 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fe60c0668d0 con 0x7fe6140d1ae0 2024-09-10T22:35:50.709 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.708+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe5f4077f70 msgr2=0x7fe5f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.708+0000 7fe61973d640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe5f4077f70 0x7fe5f407a430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc034000 tx=0x7fe5fc004730 comp rx=0 tx=0).stop 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.708+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d1ae0 msgr2=0x7fe6140cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.708+0000 7fe61973d640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d1ae0 0x7fe6140cd6e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00c730 tx=0x7fe60c00cbf0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 shutdown_connections 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe5f4077f70 0x7fe5f407a430 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe61415e090 0x7fe61415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6140d2be0 0x7fe614151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 --2- 172.21.15.66:0/623059160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe6140d1ae0 0x7fe6140cd6e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:50.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 >> 172.21.15.66:0/623059160 conn(0x7fe6140d3f10 msgr2=0x7fe61406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:50.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 shutdown_connections 2024-09-10T22:35:50.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50.709+0000 7fe61973d640 1 -- 172.21.15.66:0/623059160 wait complete. 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:35:50.722 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:35:50.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:50.723 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:50.739 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42747sec preferred_lft 42747sec 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link noprefixroute 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:35:50.740 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:f5:87:39:84:80 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::f82d:d6ff:fec7:dfcb/64 scope link 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether f2:19:d9:d3:e1:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::f019:d9ff:fed3:e1f4/64 scope link 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:f5:87:39:84:80 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5cf5:87ff:fe39:8480/64 scope link 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:35:50.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:50.742 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:35:50.803 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:35:50.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:50.805 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:35:50.861 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:35:50.862 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:35:50.862 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:35:50.862 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:35:50.862 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:35:50.862 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:35:50.862 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:35:50.862 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:35:50.862 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:35:50.862 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:35:50.863 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:35:50.951 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:35:50.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:50.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:50.990 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:35:51.006 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:35:51.006 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:35:51.007 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:35:51.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:51.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:35:51.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:51.073 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:51.129 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:35:51.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:35:51.130 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:51.186 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T22:35:51.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:51.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:51.271 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:51.271 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:51.279 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:35:51.279 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:35:51.279 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.94:3300,172.21.15.94:3301" 2024-09-10T22:35:51.279 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with new device syntax: 0@7ef137cd-a1ce-4819-b27d-b71268612f85.cephfs=/ 2024-09-10T22:35:51.279 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.66:3300/172.21.15.94:3300/172.21.15.94:3301" will pass to kernel 2024-09-10T22:35:51.280 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T22:35:51.280 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-09-10T22:35:51.280 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T22:35:51.280 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T22:35:51.280 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T22:35:51.282 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:35:51.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:35:51.282 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:35:51.308 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:35:51.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:35:51.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:35:51.411 INFO:teuthology.orchestra.run.smithi066.stdout:client4993 2024-09-10T22:35:51.411 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:35:51.411 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:35:51.429 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:35:51.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:35:51.514 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.515 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-09-10T22:35:51.516 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-09-10T22:35:51.516 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.516 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-09-10T22:35:51.516 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-09-10T22:35:51.516 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.516 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:35:51.516 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-09-10T22:35:51.517 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:35:51.518 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-09-10T22:35:51.518 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:35:51.518 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:35:51.518 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-09-10T22:35:51.518 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.518 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:35:51.518 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.519 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-09-10T22:35:51.519 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.519 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-09-10T22:35:51.519 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.520 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-09-10T22:35:51.520 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-09-10T22:35:51.520 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-09-10T22:35:51.520 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:35:51.520 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on skipped message = False 2024-09-10T22:35:51.520 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:35:51.520 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.521 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.522 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-09-10T22:35:51.523 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-09-10T22:35:51.524 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-09-10T22:35:51.524 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-09-10T22:35:51.524 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-09-10T22:35:51.524 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-09-10T22:35:51.524 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-09-10T22:35:51.524 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:35:51.524 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-09-10T22:35:51.525 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-09-10T22:35:51.525 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:35:51.525 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-09-10T22:35:51.525 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:35:51.525 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:35:51.525 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-09-10T22:35:51.526 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.526 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-09-10T22:35:51.526 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:35:51.526 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-09-10T22:35:51.526 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:35:51.526 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:35:51.527 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:35:51.527 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:35:51.527 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-09-10T22:35:51.527 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 30 2024-09-10T22:35:51.527 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-09-10T22:35:51.527 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.527 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-09-10T22:35:51.528 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:35:51.528 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:35:51.528 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-09-10T22:35:51.528 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:35:51.528 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-09-10T22:35:51.528 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-09-10T22:35:51.529 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:35:51.529 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-09-10T22:35:51.529 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-09-10T22:35:51.529 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:35:51.529 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.529 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:35:51.529 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:35:51.530 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:35:51.530 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.530 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:35:51.530 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:35:51.530 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:35:51.530 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.531 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-09-10T22:35:51.531 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:35:51.531 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.531 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:35:51.531 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-09-10T22:35:51.531 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:51.531 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-09-10T22:35:51.532 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-09-10T22:35:51.533 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-09-10T22:35:51.533 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T22:35:51.533 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T22:35:51.533 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T22:35:51.533 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T22:35:51.533 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-09-10T22:35:51.533 INFO:teuthology.orchestra.run.smithi066.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.534 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:35:51.535 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:35:51.536 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-10T22:35:51.537 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-10T22:35:51.538 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-10T22:35:51.538 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:35:51.538 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on skipped message = False 2024-09-10T22:35:51.538 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:35:51.538 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.539 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-10T22:35:51.539 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-10T22:35:51.539 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.540 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-10T22:35:51.541 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-10T22:35:51.542 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 30 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-10T22:35:51.543 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:35:51.544 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-10T22:35:51.545 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2024-09-10T22:35:51.546 INFO:teuthology.orchestra.run.smithi094.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-09-10T22:35:51.548 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:35:51.548 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:35:51.548 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:35:51.549 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:51.549 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:35:51.549 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:35:51.549 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:35:51.549 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:51.549 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:35:51.549 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:35:51.549 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:35:51.550 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:51.550 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:35:51.550 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:35:51.550 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:35:51.550 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:35:51.570 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:35:51.570 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:35:51.571 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:35:51.571 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:35:51.612 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:35:51.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:35:51.613 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:35:51.613 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:35:51.621 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:35:51.621 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:51.626 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:35:51.626 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:51.658 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:35:51.658 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:51.659 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:35:51.660 DEBUG:teuthology.parallel:result is None 2024-09-10T22:35:51.660 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:35:51.702 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:35:51.714 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:35:51.738 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:35:51.739 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:35:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 -- 172.21.15.66:0/372256271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95015e2b0 msgr2=0x7fa9500d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 --2- 172.21.15.66:0/372256271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95015e2b0 0x7fa9500d27a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9500ba1f0 tx=0x7fa93802a5f0 comp rx=0 tx=0).stop 2024-09-10T22:35:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 -- 172.21.15.66:0/372256271 shutdown_connections 2024-09-10T22:35:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 --2- 172.21.15.66:0/372256271 >> v2:172.21.15.94:3300/0 conn(0x7fa9500d2ce0 0x7fa9500b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 --2- 172.21.15.66:0/372256271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa95015e2b0 0x7fa9500d27a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 -- 172.21.15.66:0/372256271 >> 172.21.15.66:0/372256271 conn(0x7fa950074c20 msgr2=0x7fa95009d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:51.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 -- 172.21.15.66:0/372256271 shutdown_connections 2024-09-10T22:35:51.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.768+0000 7fa9551ad640 1 -- 172.21.15.66:0/372256271 wait complete. 2024-09-10T22:35:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.770+0000 7fa9551ad640 1 Processor -- start 2024-09-10T22:35:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.770+0000 7fa9551ad640 1 -- start start 2024-09-10T22:35:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa9551ad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9500d2ce0 0x7fa95009e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:51.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa9551ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9500d4000 0x7fa950158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa9551ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa95015e2b0 0x7fa950158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa9551ad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa9500b2b20 con 0x7fa9500d4000 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa9551ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9500b29a0 con 0x7fa95015e2b0 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa9551ad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa9500b2ca0 con 0x7fa9500d2ce0 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9500d4000 0x7fa950158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9500d4000 0x7fa950158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52558/0 (socket says 172.21.15.66:52558) 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 -- 172.21.15.66:0/3790754062 learned_addr learned my addr 172.21.15.66:0/3790754062 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa94f59e640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa95015e2b0 0x7fa950158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa94ed9d640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9500d2ce0 0x7fa95009e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 -- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9500d2ce0 msgr2=0x7fa95009e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9500d2ce0 0x7fa95009e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 -- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa95015e2b0 msgr2=0x7fa950158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa95015e2b0 0x7fa950158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:51.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 -- 172.21.15.66:0/3790754062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9500b9090 con 0x7fa9500d4000 2024-09-10T22:35:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa94f59e640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa95015e2b0 0x7fa950158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa947fff640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9500d4000 0x7fa950158200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa938034040 tx=0x7fa9380038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.771+0000 7fa9477fe640 1 -- 172.21.15.66:0/3790754062 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93803e030 con 0x7fa9500d4000 2024-09-10T22:35:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.772+0000 7fa9477fe640 1 -- 172.21.15.66:0/3790754062 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa938003db0 con 0x7fa9500d4000 2024-09-10T22:35:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.772+0000 7fa9477fe640 1 -- 172.21.15.66:0/3790754062 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938035040 con 0x7fa9500d4000 2024-09-10T22:35:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.772+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9500b9370 con 0x7fa9500d4000 2024-09-10T22:35:51.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.772+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9500b9930 con 0x7fa9500d4000 2024-09-10T22:35:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.773+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9500b31c0 con 0x7fa9500d4000 2024-09-10T22:35:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.773+0000 7fa9477fe640 1 -- 172.21.15.66:0/3790754062 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa938004a00 con 0x7fa9500d4000 2024-09-10T22:35:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.774+0000 7fa9477fe640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa930077f70 0x7fa93007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.774+0000 7fa9477fe640 1 -- 172.21.15.66:0/3790754062 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa9380ba890 con 0x7fa9500d4000 2024-09-10T22:35:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.774+0000 7fa94ed9d640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa930077f70 0x7fa93007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:51.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.775+0000 7fa94ed9d640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa930077f70 0x7fa93007a430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa940000f60 tx=0x7fa940009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:51.780 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:51.778+0000 7fa9477fe640 1 -- 172.21.15.66:0/3790754062 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa938082dd0 con 0x7fa9500d4000 2024-09-10T22:35:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.000+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9500d11f0 con 0x7fa9500d4000 2024-09-10T22:35:52.003 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.001+0000 7fa9477fe640 1 -- 172.21.15.66:0/3790754062 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7fa938087800 con 0x7fa9500d4000 2024-09-10T22:35:52.003 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:35:52.003 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"btime":"2024-09-10T22:35:47:657322+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/307839984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":307839984},{"type":"v1","addr":"172.21.15.94:6836","nonce":307839984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2042084416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2042084416},{"type":"v1","addr":"172.21.15.94:6837","nonce":2042084416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1521655589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1521655589},{"type":"v1","addr":"172.21.15.66:6834","nonce":1521655589}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:47.159794+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4846},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4846":{"gid":4846,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/1970715211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":1970715211},{"type":"v1","addr":"172.21.15.66:6835","nonce":1970715211}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4846,"qdb_cluster":[4846]},"id":2}]} 2024-09-10T22:35:52.004 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 17 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.003+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa930077f70 msgr2=0x7fa93007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.003+0000 7fa9551ad640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa930077f70 0x7fa93007a430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa940000f60 tx=0x7fa940009040 comp rx=0 tx=0).stop 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9500d4000 msgr2=0x7fa950158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9500d4000 0x7fa950158200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa938034040 tx=0x7fa9380038a0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 shutdown_connections 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa930077f70 0x7fa93007a430 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa95015e2b0 0x7fa950158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa9500d4000 0x7fa950158200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 --2- 172.21.15.66:0/3790754062 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa9500d2ce0 0x7fa95009e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.006 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 >> 172.21.15.66:0/3790754062 conn(0x7fa950074c20 msgr2=0x7fa9500cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 shutdown_connections 2024-09-10T22:35:52.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.004+0000 7fa9551ad640 1 -- 172.21.15.66:0/3790754062 wait complete. 2024-09-10T22:35:52.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:35:44.016061+0000', 'modified': '2024-09-10T22:35:47.159794+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4846}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4846': {'gid': 4846, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/1970715211', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 1970715211}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 1970715211}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4846, 'qdb_cluster': [4846]} 2024-09-10T22:35:52.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:35:52.421 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 -- 172.21.15.66:0/2058939309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 msgr2=0x7f5a500b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 --2- 172.21.15.66:0/2058939309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 0x7f5a500b4db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009900 tx=0x7f5a3802a610 comp rx=0 tx=0).stop 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 -- 172.21.15.66:0/2058939309 shutdown_connections 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 --2- 172.21.15.66:0/2058939309 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 0x7f5a500b4db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 --2- 172.21.15.66:0/2058939309 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a50161730 0x7f5a500b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 -- 172.21.15.66:0/2058939309 >> 172.21.15.66:0/2058939309 conn(0x7f5a5009d420 msgr2=0x7f5a500d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 -- 172.21.15.66:0/2058939309 shutdown_connections 2024-09-10T22:35:52.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.526+0000 7f5a5589b640 1 -- 172.21.15.66:0/2058939309 wait complete. 2024-09-10T22:35:52.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.527+0000 7f5a5589b640 1 Processor -- start 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.528+0000 7f5a5589b640 1 -- start start 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.528+0000 7f5a5589b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a500d12a0 0x7f5a501f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.528+0000 7f5a5589b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 0x7f5a5015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.528+0000 7f5a5589b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a50161730 0x7f5a5006e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.528+0000 7f5a5589b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a500b5680 con 0x7f5a500d12a0 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.528+0000 7f5a5589b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a500b5500 con 0x7f5a500d4840 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.528+0000 7f5a5589b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5a500cf350 con 0x7f5a50161730 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a500d12a0 0x7f5a501f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a500d12a0 0x7f5a501f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52566/0 (socket says 172.21.15.66:52566) 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 -- 172.21.15.66:0/2850952630 learned_addr learned my addr 172.21.15.66:0/2850952630 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 0x7f5a5015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.532 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4f7fe640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a50161730 0x7f5a5006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 -- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a50161730 msgr2=0x7f5a5006e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a50161730 0x7f5a5006e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 -- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 msgr2=0x7f5a5015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 0x7f5a5015a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 -- 172.21.15.66:0/2850952630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5015c2b0 con 0x7f5a500d12a0 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a500d4840 0x7f5a5015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a4effd640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a500d12a0 0x7f5a501f4d20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e9f0 tx=0x7f5a4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a47fff640 1 -- 172.21.15.66:0/2850952630 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000cd80 con 0x7f5a500d12a0 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a47fff640 1 -- 172.21.15.66:0/2850952630 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a40004020 con 0x7f5a500d12a0 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a5589b640 1 -- 172.21.15.66:0/2850952630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5015c590 con 0x7f5a500d12a0 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a47fff640 1 -- 172.21.15.66:0/2850952630 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40004470 con 0x7f5a500d12a0 2024-09-10T22:35:52.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.529+0000 7f5a5589b640 1 -- 172.21.15.66:0/2850952630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5015cb50 con 0x7f5a500d12a0 2024-09-10T22:35:52.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.531+0000 7f5a47fff640 1 -- 172.21.15.66:0/2850952630 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5a400035f0 con 0x7f5a500d12a0 2024-09-10T22:35:52.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.531+0000 7f5a47fff640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a30077f70 0x7f5a3007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.532+0000 7f5a47fff640 1 -- 172.21.15.66:0/2850952630 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5a4009a6c0 con 0x7f5a500d12a0 2024-09-10T22:35:52.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.532+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a30077f70 0x7f5a3007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.532+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2850952630 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a30077f70 0x7f5a3007a430 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a38033000 tx=0x7f5a38004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.532+0000 7f5a58669740 1 Processor -- start 2024-09-10T22:35:52.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.532+0000 7f5a58669740 1 -- start start 2024-09-10T22:35:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a58669740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d078b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a58669740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 0x55d17d0c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a58669740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0765f0 0x55d17d076a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a58669740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d17cfed820 con 0x55d17d0a11b0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a58669740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d17cfdcc50 con 0x55d17d0765f0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a58669740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55d17cfe0a60 con 0x55d17d0c0df0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d078b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0765f0 0x55d17d076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d078b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52580/0 (socket says 172.21.15.66:52580) 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a4effd640 1 -- 172.21.15.66:0/1561549989 learned_addr learned my addr 172.21.15.66:0/1561549989 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.533+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 0x55d17d0c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a4f7fe640 1 -- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 msgr2=0x55d17d0c11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a4f7fe640 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 0x55d17d0c11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a4f7fe640 1 -- 172.21.15.66:0/1561549989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 msgr2=0x55d17d078b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a4f7fe640 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d078b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a4f7fe640 1 -- 172.21.15.66:0/1561549989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d17cd3e850 con 0x55d17d0765f0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a4f7fe640 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0765f0 0x55d17d076a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5a5006d5a0 tx=0x7f5a48001630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a4e34399f5e1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a457fa640 1 -- 172.21.15.66:0/1561549989 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800fc10 con 0x55d17d0765f0 2024-09-10T22:35:52.538 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a457fa640 1 -- 172.21.15.66:0/1561549989 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a48018070 con 0x55d17d0765f0 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.534+0000 7f5a457fa640 1 -- 172.21.15.66:0/1561549989 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48003bd0 con 0x55d17d0765f0 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 -- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0765f0 msgr2=0x55d17d076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0765f0 0x55d17d076a40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5a5006d5a0 tx=0x7f5a48001630 comp rx=0 tx=0).stop 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 -- 172.21.15.66:0/1561549989 shutdown_connections 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0765f0 0x55d17d076a40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 0x55d17d0c11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 --2- 172.21.15.66:0/1561549989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d078b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 -- 172.21.15.66:0/1561549989 >> 172.21.15.66:0/1561549989 conn(0x55d17d0776a0 msgr2=0x55d17d07b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 -- 172.21.15.66:0/1561549989 shutdown_connections 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 -- 172.21.15.66:0/1561549989 wait complete. 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.535+0000 7f5a58669740 1 -- start start 2024-09-10T22:35:52.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 1 Processor -- start 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d099170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 0x55d17d0bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0ba1d0 0x55d17d0bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d17cd4d620 con 0x55d17d0a11b0 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d17cf825f0 con 0x55d17d0ba1d0 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55d17d075890 con 0x55d17d0c0df0 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a58669740 10 client.0 _ll_register_callbacks cb 0x55d17cf860a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.536+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d099170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:52588/0 (socket says 172.21.15.66:52588) 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4e7fc640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 0x55d17d0bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 -- 172.21.15.66:0/2991145933 learned_addr learned my addr 172.21.15.66:0/2991145933 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4f7fe640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0ba1d0 0x55d17d0bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 msgr2=0x55d17d0bb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d17d0c0df0 0x55d17d0bb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0ba1d0 msgr2=0x55d17d0bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d17d0ba1d0 0x55d17d0bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 -- 172.21.15.66:0/2991145933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d17cdb56b0 con 0x55d17d0a11b0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a4effd640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d17d0a11b0 0x55d17d099170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e9c0 tx=0x7f5a4005eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d091955dc987ad9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a2e7fc640 1 -- 172.21.15.66:0/2991145933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40088070 con 0x55d17d0a11b0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a2e7fc640 1 -- 172.21.15.66:0/2991145933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5a40061e60 con 0x55d17d0a11b0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a58669740 1 -- 172.21.15.66:0/2991145933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a48018070 con 0x55d17d0a11b0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a2e7fc640 1 -- 172.21.15.66:0/2991145933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5a40067630 con 0x55d17d0a11b0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a58669740 10 client.5124 fetch_fsmap learned FSMap version 17 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a58669740 1 -- 172.21.15.66:0/2991145933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d17d0a3a20 con 0x55d17d0a11b0 2024-09-10T22:35:52.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.537+0000 7f5a2e7fc640 1 -- 172.21.15.66:0/2991145933 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a48018070 con 0x55d17d0a11b0 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a2e7fc640 1 -- 172.21.15.66:0/2991145933 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5a4005e550 con 0x55d17d0a11b0 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a58669740 10 client.5124 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a58669740 10 client.5124 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a58669740 20 client.5124 populate_metadata read hostname 'smithi066' 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a58669740 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a58669740 4 client.5124 mds_command: new command op to 4846 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a58669740 1 -- 172.21.15.66:0/2991145933 --> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d17cfef770 con 0x55d17d0a2d90 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:35:52.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.538+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T22:35:52.686 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:35:52.684+0000 7fe21bcb1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:35:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.740+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.740+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:35:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.740+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:35:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.740+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:35:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:52.740+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T22:35:53.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.141+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:53.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.141+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:35:53.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.141+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:35:53.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.141+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:35:53.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.141+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T22:35:53.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.943+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:53.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.943+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:35:53.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.943+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:35:53.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.943+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:35:53.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:53.943+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T22:35:55.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:55.545+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:55.547 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:55.545+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:35:55.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:55.545+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:35:55.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:55.545+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:35:55.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:55.545+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T22:35:57.697 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:35:57.694+0000 7facada8a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:35:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:58.749+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:35:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:58.749+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:35:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:58.749+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:35:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:58.749+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:35:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:35:58.749+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-09-10T22:36:01.673 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:36:01.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:36:05.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:05.157+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:36:05.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:05.157+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:36:05.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:05.157+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:36:05.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:05.157+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:36:05.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:05.157+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-09-10T22:36:17.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:17.969+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:36:17.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:17.969+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:36:17.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:17.969+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:36:17.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:17.969+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:36:17.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:17.969+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:36:31.748 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:36:31.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:36:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:32.975+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:36:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:32.976+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:36:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:32.976+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:36:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:32.976+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:36:32.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:32.976+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:36:47.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:47.983+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:36:47.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:47.984+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:36:47.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:47.984+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:36:47.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:47.984+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:36:47.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:36:47.984+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:37:01.905 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:37:01.909 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:37:03.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:02.999+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:03.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:03.000+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:37:03.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:03.000+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:37:03.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:03.000+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:37:03.001 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:03.000+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:37:18.007 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:18.006+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:18.006+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:18.006+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:18.006+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:18.006+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:37:31.968 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:37:31.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:37:33.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:33.020+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:33.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:33.020+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:37:33.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:33.020+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:37:33.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:33.020+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:37:33.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:33.020+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:37:48.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:48.035+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:48.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:48.035+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-10T22:37:48.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:48.035+0000 7f5a4e7fc640 1 -- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 msgr2=0x55d17d0a3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:37:48.037 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:48.035+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:37:48.038 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:48.035+0000 7f5a4e7fc640 1 --2- 172.21.15.66:0/2991145933 >> [v2:172.21.15.66:6833/1970715211,v1:172.21.15.66:6835/1970715211] conn(0x55d17d0a2d90 0x55d17d0a3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:37:52.454 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-09-10T22:37:53.456 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:37:53.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.572+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3576478195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e80d4000 msgr2=0x7f11e80d43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:53.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.572+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3576478195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e80d4000 0x7f11e80d43e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009830 tx=0x7f11d802a510 comp rx=0 tx=0).stop 2024-09-10T22:37:53.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.572+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3576478195 shutdown_connections 2024-09-10T22:37:53.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.572+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3576478195 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11e80b1fb0 0x7f11e80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.575 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.572+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3576478195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e80d4000 0x7f11e80d43e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.572+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3576478195 >> 172.21.15.66:0/3576478195 conn(0x7f11e8074c20 msgr2=0x7f11e809d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.573+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3576478195 shutdown_connections 2024-09-10T22:37:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.573+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3576478195 wait complete. 2024-09-10T22:37:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.574+0000 7f11ee8dd640 1 Processor -- start 2024-09-10T22:37:53.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.574+0000 7f11ee8dd640 1 -- start start 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ee8dd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11e80b1fb0 0x7f11e8159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ee8dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e80d4000 0x7f11e814b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ee8dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e815e2b0 0x7f11e814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ee8dd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80b2b00 con 0x7f11e815e2b0 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ee8dd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80b2980 con 0x7f11e80d4000 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ee8dd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f11e80b2c80 con 0x7f11e80b1fb0 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e815e2b0 0x7f11e814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e815e2b0 0x7f11e814d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57978/0 (socket says 172.21.15.66:57978) 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 -- 172.21.15.66:0/3269342085 learned_addr learned my addr 172.21.15.66:0/3269342085 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:37:53.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11e7fff640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11e80b1fb0 0x7f11e8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11e77fe640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e80d4000 0x7f11e814b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 -- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11e80b1fb0 msgr2=0x7f11e8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11e80b1fb0 0x7f11e8159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 -- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e80d4000 msgr2=0x7f11e814b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e80d4000 0x7f11e814b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.575+0000 7f11ece7a640 1 -- 172.21.15.66:0/3269342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e814de20 con 0x7f11e815e2b0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.576+0000 7f11ece7a640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e815e2b0 0x7f11e814d8e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f11e000b9a0 tx=0x7f11e000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.576+0000 7f11e4ff9640 1 -- 172.21.15.66:0/3269342085 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0019070 con 0x7f11e815e2b0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.576+0000 7f11e4ff9640 1 -- 172.21.15.66:0/3269342085 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11e000ccb0 con 0x7f11e815e2b0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.576+0000 7f11e4ff9640 1 -- 172.21.15.66:0/3269342085 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e00146a0 con 0x7f11e815e2b0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.576+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e80cd480 con 0x7f11e815e2b0 2024-09-10T22:37:53.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.576+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e814e370 con 0x7f11e815e2b0 2024-09-10T22:37:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.577+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e80b3210 con 0x7f11e815e2b0 2024-09-10T22:37:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.577+0000 7f11e4ff9640 1 -- 172.21.15.66:0/3269342085 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f11e001d3b0 con 0x7f11e815e2b0 2024-09-10T22:37:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.578+0000 7f11e4ff9640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f11d4077f70 0x7f11d407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:53.581 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.578+0000 7f11e4ff9640 1 -- 172.21.15.66:0/3269342085 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f11e0099e20 con 0x7f11e815e2b0 2024-09-10T22:37:53.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.578+0000 7f11e7fff640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f11d4077f70 0x7f11d407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:53.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.579+0000 7f11e7fff640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f11d4077f70 0x7f11d407a430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11d8034000 tx=0x7f11d8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:53.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.582+0000 7f11e4ff9640 1 -- 172.21.15.66:0/3269342085 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f11e0062950 con 0x7f11e815e2b0 2024-09-10T22:37:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.803+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e80cece0 con 0x7f11e815e2b0 2024-09-10T22:37:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.804+0000 7f11e4ff9640 1 -- 172.21.15.66:0/3269342085 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5215 (secure 0 0 0) 0x7f11e0062250 con 0x7f11e815e2b0 2024-09-10T22:37:53.806 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:37:53.806 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"btime":"2024-09-10T22:35:56:704166+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4252749143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4252749143},{"type":"v1","addr":"172.21.15.94:6835","nonce":4252749143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/58860958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":58860958},{"type":"v1","addr":"172.21.15.94:6837","nonce":58860958}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3000446890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3000446890},{"type":"v1","addr":"172.21.15.66:6835","nonce":3000446890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:56.704154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/956430443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":956430443},{"type":"v1","addr":"172.21.15.66:6834","nonce":956430443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5106,"qdb_cluster":[5106]},"id":2}]} 2024-09-10T22:37:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-09-10T22:37:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f11d4077f70 msgr2=0x7f11d407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f11d4077f70 0x7f11d407a430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f11d8034000 tx=0x7f11d8004730 comp rx=0 tx=0).stop 2024-09-10T22:37:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e815e2b0 msgr2=0x7f11e814d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e815e2b0 0x7f11e814d8e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f11e000b9a0 tx=0x7f11e000be60 comp rx=0 tx=0).stop 2024-09-10T22:37:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 shutdown_connections 2024-09-10T22:37:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f11d4077f70 0x7f11d407a430 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f11e815e2b0 0x7f11e814d8e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11e80d4000 0x7f11e814b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 --2- 172.21.15.66:0/3269342085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f11e80b1fb0 0x7f11e8159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 >> 172.21.15.66:0/3269342085 conn(0x7f11e8074c20 msgr2=0x7f11e80d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 shutdown_connections 2024-09-10T22:37:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:53.806+0000 7f11ee8dd640 1 -- 172.21.15.66:0/3269342085 wait complete. 2024-09-10T22:37:54.222 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:35:44.016061+0000', 'modified': '2024-09-10T22:35:56.704154+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 35, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5106}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5106': {'gid': 5106, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.66:6834/956430443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 956430443}, {'type': 'v1', 'addr': '172.21.15.66:6834', 'nonce': 956430443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5106, 'qdb_cluster': [5106]} 2024-09-10T22:37:54.222 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:37:54.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:37:54.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.330+0000 7f0d253ad640 1 -- 172.21.15.66:0/3878386199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c004800 msgr2=0x7f0d0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.330+0000 7f0d253ad640 1 --2- 172.21.15.66:0/3878386199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c004800 0x7f0d0c004c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0d08006fd0 tx=0x7f0d0c005830 comp rx=0 tx=0).stop 2024-09-10T22:37:54.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.330+0000 7f0d253ad640 1 -- 172.21.15.66:0/3878386199 shutdown_connections 2024-09-10T22:37:54.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.330+0000 7f0d253ad640 1 --2- 172.21.15.66:0/3878386199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d0c004800 0x7f0d0c004c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.330+0000 7f0d253ad640 1 -- 172.21.15.66:0/3878386199 >> 172.21.15.66:0/3878386199 conn(0x7f0d2009d400 msgr2=0x7f0d200d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:54.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.330+0000 7f0d253ad640 1 -- 172.21.15.66:0/3878386199 shutdown_connections 2024-09-10T22:37:54.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.330+0000 7f0d253ad640 1 -- 172.21.15.66:0/3878386199 wait complete. 2024-09-10T22:37:54.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.331+0000 7f0d253ad640 1 Processor -- start 2024-09-10T22:37:54.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.331+0000 7f0d253ad640 1 -- start start 2024-09-10T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d253ad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d0c004800 0x7f0d20157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d253ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20157780 0x7f0d20152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d253ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d200cbb80 0x7f0d200b9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d253ad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200d2350 con 0x7f0d200cbb80 2024-09-10T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d253ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200ceea0 con 0x7f0d20157780 2024-09-10T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d253ad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0d200d24d0 con 0x7f0d0c004800 2024-09-10T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d200cbb80 0x7f0d200b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d200cbb80 0x7f0d200b9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57992/0 (socket says 172.21.15.66:57992) 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 -- 172.21.15.66:0/2631417561 learned_addr learned my addr 172.21.15.66:0/2631417561 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1effd640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d0c004800 0x7f0d20157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20157780 0x7f0d20152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 -- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d0c004800 msgr2=0x7f0d20157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d0c004800 0x7f0d20157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 -- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20157780 msgr2=0x7f0d20152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20157780 0x7f0d20152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 -- 172.21.15.66:0/2631417561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d200caf20 con 0x7f0d200cbb80 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.332+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d200cbb80 0x7f0d200b9110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d2006d580 tx=0x7f0d10001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.333+0000 7f0d177fe640 1 -- 172.21.15.66:0/2631417561 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100107d0 con 0x7f0d200cbb80 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.333+0000 7f0d177fe640 1 -- 172.21.15.66:0/2631417561 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d10016070 con 0x7f0d200cbb80 2024-09-10T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.333+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d200c7de0 con 0x7f0d200cbb80 2024-09-10T22:37:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.333+0000 7f0d177fe640 1 -- 172.21.15.66:0/2631417561 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10010ce0 con 0x7f0d200cbb80 2024-09-10T22:37:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.333+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d200cb380 con 0x7f0d200cbb80 2024-09-10T22:37:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.335+0000 7f0d177fe640 1 -- 172.21.15.66:0/2631417561 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0d1001b460 con 0x7f0d200cbb80 2024-09-10T22:37:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.335+0000 7f0d177fe640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d00077fa0 0x7f0d0007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.335+0000 7f0d177fe640 1 -- 172.21.15.66:0/2631417561 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f0d1009a510 con 0x7f0d200cbb80 2024-09-10T22:37:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.335+0000 7f0d1effd640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d00077fa0 0x7f0d0007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.336+0000 7f0d1effd640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d00077fa0 0x7f0d0007a460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d18005fd0 tx=0x7f0d18009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.336+0000 7f0d2825a740 1 Processor -- start 2024-09-10T22:37:54.338 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.336+0000 7f0d2825a740 1 -- start start 2024-09-10T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f870adf0 0x55b3f870b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f86c05f0 0x55b3f86c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d2825a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b3f8637820 con 0x55b3f86eb1b0 2024-09-10T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3f8626c50 con 0x55b3f870adf0 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55b3f862aa60 con 0x55b3f86c05f0 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58006/0 (socket says 172.21.15.66:58006) 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 -- 172.21.15.66:0/1256979209 learned_addr learned my addr 172.21.15.66:0/1256979209 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f870adf0 0x55b3f870b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f86c05f0 0x55b3f86c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 -- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f86c05f0 msgr2=0x55b3f86c0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f86c05f0 0x55b3f86c0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 -- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f870adf0 msgr2=0x55b3f870b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f870adf0 0x55b3f870b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 -- 172.21.15.66:0/1256979209 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f8388850 con 0x55b3f86eb1b0 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d1effd640 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c2b20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0d18003d50 tx=0x7f0d18016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57e3fe311336e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.337+0000 7f0d14ff9640 1 -- 172.21.15.66:0/1256979209 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800f4e0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d14ff9640 1 -- 172.21.15.66:0/1256979209 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d1800f680 con 0x55b3f86eb1b0 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d14ff9640 1 -- 172.21.15.66:0/1256979209 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800fa90 con 0x55b3f86eb1b0 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 -- 172.21.15.66:0/1256979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 msgr2=0x55b3f86c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c2b20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0d18003d50 tx=0x7f0d18016000 comp rx=0 tx=0).stop 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 -- 172.21.15.66:0/1256979209 shutdown_connections 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f86c05f0 0x55b3f86c0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f870adf0 0x55b3f870b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 --2- 172.21.15.66:0/1256979209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c2b20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 -- 172.21.15.66:0/1256979209 >> 172.21.15.66:0/1256979209 conn(0x55b3f86c16a0 msgr2=0x55b3f86c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 -- 172.21.15.66:0/1256979209 shutdown_connections 2024-09-10T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.338+0000 7f0d2825a740 1 -- 172.21.15.66:0/1256979209 wait complete. 2024-09-10T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.339+0000 7f0d2825a740 1 -- start start 2024-09-10T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.339+0000 7f0d2825a740 1 Processor -- start 2024-09-10T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f871d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f86ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f86e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b3f8397620 con 0x55b3f86eb1b0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3f85cc5f0 con 0x55b3f871a730 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55b3f86bfb30 con 0x55b3f870adf0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 10 client.0 _ll_register_callbacks cb 0x55b3f85d00a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f871d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f871d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58012/0 (socket says 172.21.15.66:58012) 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 -- 172.21.15.66:0/4170600170 learned_addr learned my addr 172.21.15.66:0/4170600170 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f86ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 -- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 msgr2=0x55b3f86ec8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f86e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f86ec8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 -- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 msgr2=0x55b3f86e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f86e32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 -- 172.21.15.66:0/4170600170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f83ff6b0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d1effd640 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f871d0b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55b3f870bc30 tx=0x7f0d1800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f041ae1d8f19359b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0cfdffb640 1 -- 172.21.15.66:0/4170600170 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800f4e0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0cfdffb640 1 -- 172.21.15.66:0/4170600170 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f0d18028680 con 0x55b3f86eb1b0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.340+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d1800f680 con 0x55b3f86eb1b0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0cfdffb640 1 -- 172.21.15.66:0/4170600170 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0d18028880 con 0x55b3f86eb1b0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 10 client.5145 fetch_fsmap learned FSMap version 24 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b3f86e5fd0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0cfdffb640 1 -- 172.21.15.66:0/4170600170 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d1800f680 con 0x55b3f86eb1b0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0cfdffb640 1 -- 172.21.15.66:0/4170600170 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f0d18030020 con 0x55b3f86eb1b0 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 10 client.5145 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 10 client.5145 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 20 client.5145 populate_metadata read hostname 'smithi066' 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86e43f0 0x55b3f86e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 4 client.5145 mds_command: new command op to 5106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 --> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3f8639770 con 0x55b3f86e43f0 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86e43f0 0x55b3f86e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0cfdffb640 10 client.5145 ms_handle_connect on v2:172.21.15.66:6832/956430443 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.341+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86e43f0 0x55b3f86e47d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6add52d92560b396 server_cookie=be74aff443c7caba in_seq=0 out_seq=0 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0cfdffb640 1 -- 172.21.15.66:0/4170600170 <== mds.0 v2:172.21.15.66:6832/956430443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3f8639770 con 0x55b3f86e43f0 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0cfdffb640 10 client.5145 handle_command_reply: tid=0 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 client.5145 shutdown 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 msgr2=0x55b3f871d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f871d0b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55b3f870bc30 tx=0x7f0d1800fc30 comp rx=0 tx=0).stop 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 shutdown_connections 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86e43f0 0x55b3f86e47d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f86e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f86ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 --2- 172.21.15.66:0/4170600170 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f871d0b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 >> 172.21.15.66:0/4170600170 conn(0x55b3f86c05f0 msgr2=0x55b3f86be5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0cfdffb640 0 client.5145 ms_handle_reset on v2:172.21.15.66:6832/956430443 2024-09-10T22:37:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 shutdown_connections 2024-09-10T22:37:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.342+0000 7f0d2825a740 1 -- 172.21.15.66:0/4170600170 wait complete. 2024-09-10T22:37:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.343+0000 7f0d2825a740 20 client.5145 trim_cache size 0 max 16384 2024-09-10T22:37:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.361+0000 7f0d2825a740 1 Processor -- start 2024-09-10T22:37:54.363 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.361+0000 7f0d2825a740 1 -- start start 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f870adf0 0x55b3f86e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f871a730 0x55b3f86c09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d2825a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20200af0 con 0x55b3f870adf0 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d201f8ac0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0d202004a0 con 0x55b3f871a730 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f870adf0 0x55b3f86e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f870adf0 0x55b3f86e31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58028/0 (socket says 172.21.15.66:58028) 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 -- 172.21.15.66:0/380295878 learned_addr learned my addr 172.21.15.66:0/380295878 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1effd640 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f871a730 0x55b3f86c09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.371 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 -- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f871a730 msgr2=0x55b3f86c09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f871a730 0x55b3f86c09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 -- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f86eb1b0 msgr2=0x55b3f86c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 -- 172.21.15.66:0/380295878 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f8704520 con 0x55b3f870adf0 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1effd640 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f871a730 0x55b3f86c09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f870adf0 0x55b3f86e31b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0d0802b0f0 tx=0x7f0d200d05c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8002c99fedfd3574 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0cfffff640 1 -- 172.21.15.66:0/380295878 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08031040 con 0x55b3f870adf0 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0cfffff640 1 -- 172.21.15.66:0/380295878 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d080016b0 con 0x55b3f870adf0 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.362+0000 7f0cfffff640 1 -- 172.21.15.66:0/380295878 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08006300 con 0x55b3f870adf0 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 -- 172.21.15.66:0/380295878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f870adf0 msgr2=0x55b3f86e31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f870adf0 0x55b3f86e31b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0d0802b0f0 tx=0x7f0d200d05c0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 -- 172.21.15.66:0/380295878 shutdown_connections 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f871a730 0x55b3f86c09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f870adf0 0x55b3f86e31b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 --2- 172.21.15.66:0/380295878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f86eb1b0 0x55b3f86c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 -- 172.21.15.66:0/380295878 >> 172.21.15.66:0/380295878 conn(0x55b3f86c05f0 msgr2=0x55b3f872dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 -- 172.21.15.66:0/380295878 shutdown_connections 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 -- 172.21.15.66:0/380295878 wait complete. 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 -- start start 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.363+0000 7f0d2825a740 1 Processor -- start 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f878a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f8701b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d2825a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f8702040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d2825a740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55b3f86e38a0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55b3f86ecf50 con 0x55b3f871a730 2024-09-10T22:37:54.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d2825a740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55b3f86e6c90 con 0x55b3f870adf0 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d2825a740 10 client.0 _ll_register_callbacks cb 0x7f0d180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f878a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d1effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f878a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58034/0 (socket says 172.21.15.66:58034) 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d1effd640 1 -- 172.21.15.66:0/3966742757 learned_addr learned my addr 172.21.15.66:0/3966742757 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f8701b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.364+0000 7f0d1f7fe640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f8702040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d1effd640 1 -- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 msgr2=0x55b3f8701b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d1effd640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f8701b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d1effd640 1 -- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 msgr2=0x55b3f8702040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d1effd640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f8702040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d1effd640 1 -- 172.21.15.66:0/3966742757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f87026d0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.374 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f8701b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d1effd640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f878a950 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800ff30 tx=0x7f0d180286b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f071527d7e1601b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0cfdffb640 1 -- 172.21.15.66:0/3966742757 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18028e30 con 0x55b3f86eb1b0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0cfdffb640 1 -- 172.21.15.66:0/3966742757 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f0d1803fb40 con 0x55b3f86eb1b0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d080016b0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0cfdffb640 1 -- 172.21.15.66:0/3966742757 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0d18003c50 con 0x55b3f86eb1b0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 10 client.5151 fetch_fsmap learned FSMap version 24 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55b3f86dfc00 con 0x55b3f86eb1b0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0cfdffb640 1 -- 172.21.15.66:0/3966742757 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d080016b0 con 0x55b3f86eb1b0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0cfdffb640 1 -- 172.21.15.66:0/3966742757 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 24) ==== 3502+0+0 (secure 0 0 0) 0x7f0d18030070 con 0x55b3f86eb1b0 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 10 client.5151 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 10 client.5151 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 20 client.5151 populate_metadata read hostname 'smithi066' 2024-09-10T22:37:54.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86df600 0x55b3f872c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 4 client.5151 mds_command: new command op to 5106 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.365+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 --> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] -- command(tid 0: {"prefix": "status"}) -- 0x55b3f8635220 con 0x55b3f86df600 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86df600 0x55b3f872c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0cfdffb640 10 client.5151 ms_handle_connect on v2:172.21.15.66:6832/956430443 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d1e7fc640 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86df600 0x55b3f872c560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f41d76f7a722de server_cookie=371476e3e4c1c5b4 in_seq=0 out_seq=0 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0cfdffb640 1 -- 172.21.15.66:0/3966742757 <== mds.0 v2:172.21.15.66:6832/956430443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b3f8635220 con 0x55b3f86df600 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0cfdffb640 10 client.5151 handle_command_reply: tid=0 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 client.5151 shutdown 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 msgr2=0x55b3f878a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f878a950 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800ff30 tx=0x7f0d180286b0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 shutdown_connections 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:6832/956430443,v1:172.21.15.66:6834/956430443] conn(0x55b3f86df600 0x55b3f872c560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b3f871a730 0x55b3f8702040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55b3f870adf0 0x55b3f8701b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 --2- 172.21.15.66:0/3966742757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b3f86eb1b0 0x55b3f878a950 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 >> 172.21.15.66:0/3966742757 conn(0x55b3f86c05f0 msgr2=0x55b3f86c09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0cfdffb640 0 client.5151 ms_handle_reset on v2:172.21.15.66:6832/956430443 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 shutdown_connections 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 1 -- 172.21.15.66:0/3966742757 wait complete. 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.366+0000 7f0d2825a740 20 client.5151 trim_cache size 0 max 16384 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "7ef137cd-a1ce-4819-b27d-b71268612f85", 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:37:54.377 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5106, 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 121.672644797, 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 24, 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 36, 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 36, 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 122.62803461999999 2024-09-10T22:37:54.378 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d00077fa0 msgr2=0x7f0d0007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d00077fa0 0x7f0d0007a460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d18005fd0 tx=0x7f0d18009500 comp rx=0 tx=0).stop 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d200cbb80 msgr2=0x7f0d200b9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d200cbb80 0x7f0d200b9110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0d2006d580 tx=0x7f0d10001630 comp rx=0 tx=0).stop 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 shutdown_connections 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d00077fa0 0x7f0d0007a460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0d200cbb80 0x7f0d200b9110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.379 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20157780 0x7f0d20152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 --2- 172.21.15.66:0/2631417561 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0d0c004800 0x7f0d20157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:37:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 >> 172.21.15.66:0/2631417561 conn(0x7f0d2009d400 msgr2=0x7f0d2015c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:37:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 shutdown_connections 2024-09-10T22:37:54.380 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:37:54.367+0000 7f0d253ad640 1 -- 172.21.15.66:0/2631417561 wait complete. 2024-09-10T22:37:54.381 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:54.381 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:37:54.398 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file0", '"'"'w'"'"') as f: 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:37:54.399 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:37:54.441 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-09-10T22:37:54.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:37:54.457 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file0': No such file or directory 2024-09-10T22:37:54.510 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:55.458 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-09-10T22:37:55.476 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:37:55.476 INFO:tasks.cephfs.mount:File open_file0 became visible with size 7 from 0 after 1s 2024-09-10T22:37:55.476 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:55.476 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:55.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:37:55.535 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:37:55.607 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:37:55.607 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:37:55.608 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121298 s, 86.4 MB/s 2024-09-10T22:37:55.609 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:55.609 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file1", '"'"'w'"'"') as f: 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:37:55.667 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:37:55.668 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:37:55.709 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-09-10T22:37:55.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:37:55.726 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file1': No such file or directory 2024-09-10T22:37:55.778 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:56.512 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:56.727 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-09-10T22:37:56.748 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:37:56.748 INFO:tasks.cephfs.mount:File open_file1 became visible with size 7 from 0 after 1s 2024-09-10T22:37:56.748 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:56.748 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:56.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:37:56.807 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:37:56.877 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:37:56.877 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:37:56.878 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970864 s, 108 MB/s 2024-09-10T22:37:56.878 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:56.878 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:56.937 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:37:56.937 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:37:56.937 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:37:56.937 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file2", '"'"'w'"'"') as f: 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:37:56.938 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:37:56.980 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-09-10T22:37:56.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:37:56.998 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file2': No such file or directory 2024-09-10T22:37:57.051 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:57.780 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:57.999 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-09-10T22:37:58.019 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:37:58.019 INFO:tasks.cephfs.mount:File open_file2 became visible with size 7 from 0 after 1s 2024-09-10T22:37:58.020 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:58.020 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:58.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:37:58.079 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:37:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:37:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:37:58.153 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977513 s, 107 MB/s 2024-09-10T22:37:58.154 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:58.154 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:58.212 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:37:58.212 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file3", '"'"'w'"'"') as f: 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:37:58.213 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:37:58.214 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:37:58.214 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:37:58.214 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:37:58.256 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-09-10T22:37:58.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:37:58.274 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file3': No such file or directory 2024-09-10T22:37:58.327 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:58.514 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:59.054 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:59.275 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-09-10T22:37:59.296 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:37:59.296 INFO:tasks.cephfs.mount:File open_file3 became visible with size 7 from 0 after 1s 2024-09-10T22:37:59.296 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:59.297 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:59.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:37:59.357 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:37:59.434 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:37:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:37:59.435 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012981 s, 80.8 MB/s 2024-09-10T22:37:59.436 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:37:59.436 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file4", '"'"'w'"'"') as f: 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:37:59.495 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:37:59.496 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:37:59.496 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:37:59.496 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:37:59.496 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:37:59.496 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:37:59.496 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:37:59.496 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:37:59.537 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-09-10T22:37:59.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:37:59.555 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file4': No such file or directory 2024-09-10T22:37:59.609 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:37:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:00.330 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:00.555 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-09-10T22:38:00.575 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:38:00.576 INFO:tasks.cephfs.mount:File open_file4 became visible with size 7 from 0 after 1s 2024-09-10T22:38:00.576 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:00.576 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:00.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:00.635 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:38:00.711 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:38:00.711 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:38:00.711 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987147 s, 106 MB/s 2024-09-10T22:38:00.712 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:00.712 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:00.769 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file5", '"'"'w'"'"') as f: 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:38:00.770 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:38:00.771 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:38:00.771 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:38:00.771 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:38:00.771 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:38:00.812 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-09-10T22:38:00.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:38:00.829 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file5': No such file or directory 2024-09-10T22:38:00.881 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:01.055 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:01.611 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:01.784 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:01.830 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-09-10T22:38:01.850 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:38:01.851 INFO:tasks.cephfs.mount:File open_file5 became visible with size 7 from 0 after 1s 2024-09-10T22:38:01.851 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:01.851 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:01.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:01.911 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:38:01.986 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:38:01.986 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:38:01.986 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113475 s, 92.4 MB/s 2024-09-10T22:38:01.986 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:01.986 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:02.029 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:38:02.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file6", '"'"'w'"'"') as f: 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:38:02.047 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:38:02.048 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:38:02.048 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:38:02.048 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:38:02.048 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:38:02.048 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:38:02.048 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:38:02.048 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:38:02.089 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-09-10T22:38:02.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:38:02.107 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file6': No such file or directory 2024-09-10T22:38:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:02.332 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:02.883 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:03.058 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:03.107 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-09-10T22:38:03.128 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:38:03.128 INFO:tasks.cephfs.mount:File open_file6 became visible with size 7 from 0 after 1s 2024-09-10T22:38:03.128 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:03.128 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:03.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:03.187 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:38:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:38:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:38:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904618 s, 116 MB/s 2024-09-10T22:38:03.261 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:03.261 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:03.319 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:38:03.319 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:38:03.319 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/open_file7", '"'"'w'"'"') as f: 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:38:03.320 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:38:03.321 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:38:03.321 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:38:03.321 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:38:03.363 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-09-10T22:38:03.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:38:03.381 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/open_file7': No such file or directory 2024-09-10T22:38:03.435 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:03.613 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:04.161 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:04.332 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:04.382 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-09-10T22:38:04.402 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:38:04.402 INFO:tasks.cephfs.mount:File open_file7 became visible with size 7 from 0 after 1s 2024-09-10T22:38:04.402 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:04.402 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:04.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:04.462 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-10T22:38:04.519 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T22:38:04.535 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T22:38:04.536 INFO:teuthology.orchestra.run.smithi066.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975249 s, 108 MB/s 2024-09-10T22:38:04.884 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:05.437 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:05.615 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:05.789 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:06.163 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:06.334 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:06.522 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:06.886 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:07.062 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:07.440 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:07.791 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:08.165 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:08.336 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:08.524 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:09.064 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:09.442 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:09.620 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:09.793 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:10.167 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:10.338 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:10.526 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:11.066 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:11.444 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:11.622 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:11.795 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:12.170 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:12.340 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:12.527 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:12.892 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:13.446 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:13.624 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:13.797 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:14.342 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:14.529 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:14.538 DEBUG:teuthology.orchestra.run.smithi066:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.1 2024-09-10T22:38:14.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.588+0000 7ffa944b0f40 1 Processor -- start 2024-09-10T22:38:14.590 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.588+0000 7ffa944b0f40 1 -- start start 2024-09-10T22:38:14.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa944b0f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf05470e0 0x55abf065dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa944b0f40 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf0529b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa944b0f40 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf052a0d0 0x55abf06662a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa944b0f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abf044dd60 con 0x55abf05470e0 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa944b0f40 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55abf04ac430 con 0x55abf05474c0 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa944b0f40 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55abf05d4510 con 0x55abf052a0d0 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf05470e0 0x55abf065dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf05470e0 0x55abf065dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36266/0 (socket says 172.21.15.66:36266) 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 -- 172.21.15.66:0/97952725 learned_addr learned my addr 172.21.15.66:0/97952725 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa9279d640 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf052a0d0 0x55abf06662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa9179b640 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf0529b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 -- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf052a0d0 msgr2=0x55abf06662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf052a0d0 0x55abf06662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 -- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 msgr2=0x55abf0529b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:14.593 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf0529b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 -- 172.21.15.66:0/97952725 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf05d2270 con 0x55abf05470e0 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa9179b640 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf0529b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa9279d640 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf052a0d0 0x55abf06662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa91f9c640 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf05470e0 0x55abf065dff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa84000fb0 tx=0x7ffa84009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a01dc8cf89f321 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.589+0000 7ffa83fff640 1 -- 172.21.15.66:0/97952725 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400d800 con 0x55abf05470e0 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa83fff640 1 -- 172.21.15.66:0/97952725 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa84014020 con 0x55abf05470e0 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa83fff640 1 -- 172.21.15.66:0/97952725 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400dc60 con 0x55abf05470e0 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 -- 172.21.15.66:0/97952725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf05470e0 msgr2=0x55abf065dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf05470e0 0x55abf065dff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa84000fb0 tx=0x7ffa84009ac0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 -- 172.21.15.66:0/97952725 shutdown_connections 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf052a0d0 0x55abf06662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf0529b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/97952725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf05470e0 0x55abf065dff0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.594 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 -- 172.21.15.66:0/97952725 >> 172.21.15.66:0/97952725 conn(0x55abf052b910 msgr2=0x55abf053eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 -- 172.21.15.66:0/97952725 shutdown_connections 2024-09-10T22:38:14.595 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.590+0000 7ffa944b0f40 1 -- 172.21.15.66:0/97952725 wait complete. 2024-09-10T22:38:14.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.591+0000 7ffa944b0f40 1 Processor -- start 2024-09-10T22:38:14.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.591+0000 7ffa944b0f40 1 -- start start 2024-09-10T22:38:14.597 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa944b0f40 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf052a0d0 0x55abf05d0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa944b0f40 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf05470e0 0x55abf066ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa944b0f40 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf05b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa944b0f40 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55abf0529650 con 0x55abf052a0d0 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa944b0f40 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55abf0500f90 con 0x55abf05474c0 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa944b0f40 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55abf065e530 con 0x55abf05470e0 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf052a0d0 0x55abf05d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf052a0d0 0x55abf05d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36280/0 (socket says 172.21.15.66:36280) 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 -- 172.21.15.66:0/1096585260 learned_addr learned my addr 172.21.15.66:0/1096585260 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa9179b640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf05470e0 0x55abf066ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa9279d640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf05b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 -- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf05470e0 msgr2=0x55abf066ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf05470e0 0x55abf066ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 -- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 msgr2=0x55abf05b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:14.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf05b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 -- 172.21.15.66:0/1096585260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf066bf50 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa9279d640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55abf05474c0 0x55abf05b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa9179b640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55abf05470e0 0x55abf066ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.592+0000 7ffa91f9c640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf052a0d0 0x55abf05d0960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400d5c0 tx=0x7ffa840106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.593+0000 7ffa82ffd640 1 -- 172.21.15.66:0/1096585260 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84013040 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.593+0000 7ffa82ffd640 1 -- 172.21.15.66:0/1096585260 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa84010dd0 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.593+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55abf0500630 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.593+0000 7ffa82ffd640 1 -- 172.21.15.66:0/1096585260 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840256e0 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.593+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abf05b5eb0 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.594+0000 7ffa82ffd640 1 -- 172.21.15.66:0/1096585260 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffa8402d030 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.595+0000 7ffa82ffd640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa74076950 0x7ffa74078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.595+0000 7ffa82ffd640 1 -- 172.21.15.66:0/1096585260 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7ffa8409a770 con 0x55abf052a0d0 2024-09-10T22:38:14.599 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.595+0000 7ffa9179b640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa74076950 0x7ffa74078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.595+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x55abf053fd80 0x55abf05b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.595+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 --> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] -- osd_op(unknown.0.0:1 4.13 4:c9647462:::mds0_openfiles.1:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x55abf05b7640 con 0x55abf053fd80 2024-09-10T22:38:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.596+0000 7ffa9279d640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x55abf053fd80 0x55abf05b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.596+0000 7ffa9179b640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa74076950 0x7ffa74078e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa880045f0 tx=0x7ffa88004880 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.596+0000 7ffa9279d640 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x55abf053fd80 0x55abf05b7100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:14.600 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.596+0000 7ffa9279d640 1 -- 172.21.15.66:0/1096585260 <== osd.4 v2:172.21.15.94:6818/3674153549 1 ==== osd_op_reply(1 mds0_openfiles.1 [stat out=16b] v0'0 uv9 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7ffa8c00b040 con 0x55abf053fd80 2024-09-10T22:38:14.601 INFO:teuthology.orchestra.run.smithi066.stdout:cephfs_metadata/mds0_openfiles.1 mtime 2024-09-10T22:38:04.000000+0000, size 0 2024-09-10T22:38:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x55abf053fd80 msgr2=0x55abf05b7100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x55abf053fd80 0x55abf05b7100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa74076950 msgr2=0x7ffa74078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa74076950 0x7ffa74078e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffa880045f0 tx=0x7ffa88004880 comp rx=0 tx=0).stop 2024-09-10T22:38:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf052a0d0 msgr2=0x55abf05d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 --2- 172.21.15.66:0/1096585260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55abf052a0d0 0x55abf05d0960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400d5c0 tx=0x7ffa840106c0 comp rx=0 tx=0).stop 2024-09-10T22:38:14.602 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa9279d640 1 -- 172.21.15.66:0/1096585260 reap_dead start 2024-09-10T22:38:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 shutdown_connections 2024-09-10T22:38:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 >> 172.21.15.66:0/1096585260 conn(0x55abf052b910 msgr2=0x55abf052a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 shutdown_connections 2024-09-10T22:38:14.603 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14.597+0000 7ffa944b0f40 1 -- 172.21.15.66:0/1096585260 wait complete. 2024-09-10T22:38:14.603 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:38:14.603 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:38:14.648 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:38:14.696 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:38:14.696 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:38:14.697 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file0", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:14.698 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:14 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:14.894 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:15.448 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:15.626 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:15.799 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:16.174 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:16.344 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:16.531 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:16.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:16.545 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file1", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:16.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:16 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:16.895 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:17.073 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:17.450 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:17.628 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:17.801 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:17.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:17.815 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file2", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:17.856 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:17 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:18.176 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:18.346 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:18.897 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:19.075 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:19.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:19.089 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file3", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:19.130 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:19 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:19.452 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:19.630 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:20.178 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:20.349 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:20.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:20.362 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file4", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:20.403 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:20 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:20.898 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:21.454 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:21.632 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:21.637 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:21.646 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file5", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:21.688 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:21 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:22.180 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:22.900 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:22.905 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:22.914 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file6", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:22.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:22 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:23.457 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:24.182 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:24.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:24.196 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/open_file7", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:38:24.238 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:24 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:38:25.459 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:38:25.464 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:25.473 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:25.473 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:25.533 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:25.533 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:25.593 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:38:25.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 -- 172.21.15.66:0/352864157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 msgr2=0x7f46ac15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:25.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 --2- 172.21.15.66:0/352864157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac15c0f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0cd060 tx=0x7f46a802a560 comp rx=0 tx=0).stop 2024-09-10T22:38:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 -- 172.21.15.66:0/352864157 shutdown_connections 2024-09-10T22:38:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 --2- 172.21.15.66:0/352864157 >> v2:172.21.15.94:3300/0 conn(0x7f46ac0733a0 0x7f46ac06f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 --2- 172.21.15.66:0/352864157 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46ac0cee90 0x7f46ac072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 --2- 172.21.15.66:0/352864157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac15c0f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 -- 172.21.15.66:0/352864157 >> 172.21.15.66:0/352864157 conn(0x7f46ac0cfd00 msgr2=0x7f46ac0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 -- 172.21.15.66:0/352864157 shutdown_connections 2024-09-10T22:38:25.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.747+0000 7f46b3705640 1 -- 172.21.15.66:0/352864157 wait complete. 2024-09-10T22:38:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.748+0000 7f46b3705640 1 Processor -- start 2024-09-10T22:38:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.748+0000 7f46b3705640 1 -- start start 2024-09-10T22:38:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b3705640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b3705640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46ac0733a0 0x7f46ac145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b3705640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46ac0cee90 0x7f46ac14c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:25.752 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b3705640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac06fa10 con 0x7f46ac072a80 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b3705640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac06f890 con 0x7f46ac0cee90 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b3705640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f46ac06fb90 con 0x7f46ac0733a0 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac1f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34370/0 (socket says 172.21.15.66:34370) 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 -- 172.21.15.66:0/3140886573 learned_addr learned my addr 172.21.15.66:0/3140886573 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b0ca0640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46ac0733a0 0x7f46ac145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b1ca2640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46ac0cee90 0x7f46ac14c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 -- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46ac0733a0 msgr2=0x7f46ac145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46ac0733a0 0x7f46ac145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 -- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46ac0cee90 msgr2=0x7f46ac14c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46ac0cee90 0x7f46ac14c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b14a1640 1 -- 172.21.15.66:0/3140886573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac145cb0 con 0x7f46ac072a80 2024-09-10T22:38:25.753 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.749+0000 7f46b1ca2640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46ac0cee90 0x7f46ac14c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.750+0000 7f46b14a1640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac1f4840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46a80099c0 tx=0x7f46a80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.750+0000 7f46a1ffb640 1 -- 172.21.15.66:0/3140886573 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a803f070 con 0x7f46ac072a80 2024-09-10T22:38:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.750+0000 7f46a1ffb640 1 -- 172.21.15.66:0/3140886573 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a8004890 con 0x7f46ac072a80 2024-09-10T22:38:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.750+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac145f40 con 0x7f46ac072a80 2024-09-10T22:38:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.750+0000 7f46a1ffb640 1 -- 172.21.15.66:0/3140886573 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8035030 con 0x7f46ac072a80 2024-09-10T22:38:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.750+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac146500 con 0x7f46ac072a80 2024-09-10T22:38:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.751+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac070130 con 0x7f46ac072a80 2024-09-10T22:38:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.752+0000 7f46a1ffb640 1 -- 172.21.15.66:0/3140886573 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f46a8004ce0 con 0x7f46ac072a80 2024-09-10T22:38:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.752+0000 7f46a1ffb640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4690077f70 0x7f469007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.752+0000 7f46a1ffb640 1 -- 172.21.15.66:0/3140886573 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f46a80baf50 con 0x7f46ac072a80 2024-09-10T22:38:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.752+0000 7f46b0ca0640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4690077f70 0x7f469007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.753+0000 7f46b0ca0640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4690077f70 0x7f469007a430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0ab6a0 tx=0x7f469c009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.756+0000 7f46a1ffb640 1 -- 172.21.15.66:0/3140886573 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f46a8083430 con 0x7f46ac072a80 2024-09-10T22:38:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.955+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46ac1466d0 con 0x7f46ac072a80 2024-09-10T22:38:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.956+0000 7f46a1ffb640 1 -- 172.21.15.66:0/3140886573 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) ==== 80+0+122 (secure 0 0 0) 0x7f46a8087e60 con 0x7f46ac072a80 2024-09-10T22:38:25.958 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6835/1970715211 2024-09-11T22:35:52.685163+0000 2024-09-10T22:38:25.958 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6833/1970715211 2024-09-11T22:35:52.685163+0000 2024-09-10T22:38:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4690077f70 msgr2=0x7f469007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4690077f70 0x7f469007a430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0ab6a0 tx=0x7f469c009060 comp rx=0 tx=0).stop 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 msgr2=0x7f46ac1f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac1f4840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46a80099c0 tx=0x7f46a80039e0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 shutdown_connections 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4690077f70 0x7f469007a430 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46ac0cee90 0x7f46ac14c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f46ac0733a0 0x7f46ac145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 --2- 172.21.15.66:0/3140886573 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f46ac072a80 0x7f46ac1f4840 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 >> 172.21.15.66:0/3140886573 conn(0x7f46ac0cfd00 msgr2=0x7f46ac158820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 shutdown_connections 2024-09-10T22:38:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:25.958+0000 7f46b3705640 1 -- 172.21.15.66:0/3140886573 wait complete. 2024-09-10T22:38:25.971 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T22:38:25.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:25.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T22:38:26.035 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:38:26.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:26.036 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:38:26.060 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:38:26.060 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:38:26.060 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:26.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:38:26.077 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:38:26.077 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:26.077 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:26.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=930 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:38:26.118 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:38:26.118 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:38:26.118 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:26.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:38:26.135 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:26.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.260+0000 7ff45959f640 1 -- 172.21.15.66:0/2141796099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4540d0750 msgr2=0x7ff4540d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.260+0000 7ff45959f640 1 --2- 172.21.15.66:0/2141796099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4540d0750 0x7ff4540d0b60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005fd0 tx=0x7ff44c02a940 comp rx=0 tx=0).stop 2024-09-10T22:38:26.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.260+0000 7ff45959f640 1 -- 172.21.15.66:0/2141796099 shutdown_connections 2024-09-10T22:38:26.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.260+0000 7ff45959f640 1 --2- 172.21.15.66:0/2141796099 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4540d0750 0x7ff4540d0b60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.260+0000 7ff45959f640 1 -- 172.21.15.66:0/2141796099 >> 172.21.15.66:0/2141796099 conn(0x7ff4540d4010 msgr2=0x7ff4540d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:26.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.261+0000 7ff45959f640 1 -- 172.21.15.66:0/2141796099 shutdown_connections 2024-09-10T22:38:26.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.261+0000 7ff45959f640 1 -- 172.21.15.66:0/2141796099 wait complete. 2024-09-10T22:38:26.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.262+0000 7ff45959f640 1 Processor -- start 2024-09-10T22:38:26.264 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.262+0000 7ff45959f640 1 -- start start 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4540d0750 0x7ff4540ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4540d4b00 0x7ff454157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff454161510 0x7ff4541f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540c7ef0 con 0x7ff454161510 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540c7d70 con 0x7ff4540d4b00 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff4540c8070 con 0x7ff4540d0750 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff454161510 0x7ff4541f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff454161510 0x7ff4541f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34386/0 (socket says 172.21.15.66:34386) 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 -- 172.21.15.66:0/2419957342 learned_addr learned my addr 172.21.15.66:0/2419957342 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff453fff640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4540d0750 0x7ff4540ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff44bfff640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4540d4b00 0x7ff454157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 -- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4540d0750 msgr2=0x7ff4540ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4540d0750 0x7ff4540ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 -- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4540d4b00 msgr2=0x7ff454157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4540d4b00 0x7ff454157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 -- 172.21.15.66:0/2419957342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45415bdc0 con 0x7ff454161510 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff458d9e640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff454161510 0x7ff4541f7cf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009cd0 tx=0x7ff44c031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff4517fa640 1 -- 172.21.15.66:0/2419957342 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c004240 con 0x7ff454161510 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff4517fa640 1 -- 172.21.15.66:0/2419957342 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff44c0043e0 con 0x7ff454161510 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45415c060 con 0x7ff454161510 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff4517fa640 1 -- 172.21.15.66:0/2419957342 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c004830 con 0x7ff454161510 2024-09-10T22:38:26.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.263+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4541f8ac0 con 0x7ff454161510 2024-09-10T22:38:26.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.264+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4540c8570 con 0x7ff454161510 2024-09-10T22:38:26.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.265+0000 7ff4517fa640 1 -- 172.21.15.66:0/2419957342 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff44c02f070 con 0x7ff454161510 2024-09-10T22:38:26.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.266+0000 7ff4517fa640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff434077f70 0x7ff43407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.266+0000 7ff4517fa640 1 -- 172.21.15.66:0/2419957342 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7ff44c0bb360 con 0x7ff454161510 2024-09-10T22:38:26.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.266+0000 7ff453fff640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff434077f70 0x7ff43407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.267+0000 7ff453fff640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff434077f70 0x7ff43407a430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff444000f60 tx=0x7ff444009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:26.272 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.270+0000 7ff4517fa640 1 -- 172.21.15.66:0/2419957342 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff44c083840 con 0x7ff454161510 2024-09-10T22:38:26.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.489+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4540c8820 con 0x7ff454161510 2024-09-10T22:38:26.491 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.489+0000 7ff4517fa640 1 -- 172.21.15.66:0/2419957342 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5215 (secure 0 0 0) 0x7ff44c088270 con 0x7ff454161510 2024-09-10T22:38:26.491 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:26.492 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"btime":"2024-09-10T22:35:56:704166+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4252749143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4252749143},{"type":"v1","addr":"172.21.15.94:6835","nonce":4252749143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/58860958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":58860958},{"type":"v1","addr":"172.21.15.94:6837","nonce":58860958}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3000446890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3000446890},{"type":"v1","addr":"172.21.15.66:6835","nonce":3000446890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:56.704154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/956430443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":956430443},{"type":"v1","addr":"172.21.15.66:6834","nonce":956430443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5106,"qdb_cluster":[5106]},"id":2}]} 2024-09-10T22:38:26.492 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-09-10T22:38:26.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff434077f70 msgr2=0x7ff43407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff434077f70 0x7ff43407a430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff444000f60 tx=0x7ff444009040 comp rx=0 tx=0).stop 2024-09-10T22:38:26.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff454161510 msgr2=0x7ff4541f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff454161510 0x7ff4541f7cf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009cd0 tx=0x7ff44c031e20 comp rx=0 tx=0).stop 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 shutdown_connections 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff434077f70 0x7ff43407a430 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff454161510 0x7ff4541f7cf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4540d4b00 0x7ff454157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 --2- 172.21.15.66:0/2419957342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff4540d0750 0x7ff4540ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 >> 172.21.15.66:0/2419957342 conn(0x7ff4540d4010 msgr2=0x7ff4540ca1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 shutdown_connections 2024-09-10T22:38:26.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.491+0000 7ff45959f640 1 -- 172.21.15.66:0/2419957342 wait complete. 2024-09-10T22:38:26.505 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:26.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.608+0000 7f7c6909b640 1 -- 172.21.15.66:0/2525242798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 msgr2=0x7f7c640c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.608+0000 7f7c6909b640 1 --2- 172.21.15.66:0/2525242798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 0x7f7c640c57f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c005fd0 tx=0x7f7c5c02a940 comp rx=0 tx=0).stop 2024-09-10T22:38:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.608+0000 7f7c6909b640 1 -- 172.21.15.66:0/2525242798 shutdown_connections 2024-09-10T22:38:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.608+0000 7f7c6909b640 1 --2- 172.21.15.66:0/2525242798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 0x7f7c640c57f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.608+0000 7f7c6909b640 1 -- 172.21.15.66:0/2525242798 >> 172.21.15.66:0/2525242798 conn(0x7f7c640b1950 msgr2=0x7f7c640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.608+0000 7f7c6909b640 1 -- 172.21.15.66:0/2525242798 shutdown_connections 2024-09-10T22:38:26.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.608+0000 7f7c6909b640 1 -- 172.21.15.66:0/2525242798 wait complete. 2024-09-10T22:38:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.610+0000 7f7c6909b640 1 Processor -- start 2024-09-10T22:38:26.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.610+0000 7f7c6909b640 1 -- start start 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.610+0000 7f7c6909b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c64070030 0x7f7c64155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.610+0000 7f7c6909b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c64155460 0x7f7c641f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6909b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 0x7f7c640c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6909b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7c640ce940 con 0x7f7c64155460 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6909b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c640ce7c0 con 0x7f7c6415be00 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6909b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7c640ceac0 con 0x7f7c64070030 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c64155460 0x7f7c641f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c64155460 0x7f7c641f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34388/0 (socket says 172.21.15.66:34388) 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 -- 172.21.15.66:0/912673843 learned_addr learned my addr 172.21.15.66:0/912673843 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c62d9d640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c64070030 0x7f7c64155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6359e640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 0x7f7c640c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 -- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c64070030 msgr2=0x7f7c64155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c64070030 0x7f7c64155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 -- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 msgr2=0x7f7c640c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 0x7f7c640c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 -- 172.21.15.66:0/912673843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c64158210 con 0x7f7c64155460 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6259c640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c64155460 0x7f7c641f52d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c490 tx=0x7f7c5800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c4b7fe640 1 -- 172.21.15.66:0/912673843 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58016020 con 0x7f7c64155460 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c4b7fe640 1 -- 172.21.15.66:0/912673843 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c58012a00 con 0x7f7c64155460 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c641584f0 con 0x7f7c64155460 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c4b7fe640 1 -- 172.21.15.66:0/912673843 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58011380 con 0x7f7c64155460 2024-09-10T22:38:26.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.611+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6415c4f0 con 0x7f7c64155460 2024-09-10T22:38:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.612+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c640cef70 con 0x7f7c64155460 2024-09-10T22:38:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.613+0000 7f7c4b7fe640 1 -- 172.21.15.66:0/912673843 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7c58011520 con 0x7f7c64155460 2024-09-10T22:38:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.614+0000 7f7c4b7fe640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c50077f70 0x7f7c5007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.614+0000 7f7c4b7fe640 1 -- 172.21.15.66:0/912673843 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f7c58099a80 con 0x7f7c64155460 2024-09-10T22:38:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.614+0000 7f7c62d9d640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c50077f70 0x7f7c5007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.614+0000 7f7c62d9d640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c50077f70 0x7f7c5007a430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c6415c420 tx=0x7f7c54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:26.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.618+0000 7f7c4b7fe640 1 -- 172.21.15.66:0/912673843 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7c58020070 con 0x7f7c64155460 2024-09-10T22:38:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.839+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c6409e110 con 0x7f7c64155460 2024-09-10T22:38:26.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.839+0000 7f7c4b7fe640 1 -- 172.21.15.66:0/912673843 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5215 (secure 0 0 0) 0x7f7c580625b0 con 0x7f7c64155460 2024-09-10T22:38:26.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:26.842 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"btime":"2024-09-10T22:35:56:704166+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4252749143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4252749143},{"type":"v1","addr":"172.21.15.94:6835","nonce":4252749143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/58860958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":58860958},{"type":"v1","addr":"172.21.15.94:6837","nonce":58860958}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3000446890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3000446890},{"type":"v1","addr":"172.21.15.66:6835","nonce":3000446890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:56.704154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/956430443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":956430443},{"type":"v1","addr":"172.21.15.66:6834","nonce":956430443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5106,"qdb_cluster":[5106]},"id":2}]} 2024-09-10T22:38:26.842 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-09-10T22:38:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c50077f70 msgr2=0x7f7c5007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c50077f70 0x7f7c5007a430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c6415c420 tx=0x7f7c54009040 comp rx=0 tx=0).stop 2024-09-10T22:38:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c64155460 msgr2=0x7f7c641f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c64155460 0x7f7c641f52d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800c490 tx=0x7f7c5800c950 comp rx=0 tx=0).stop 2024-09-10T22:38:26.844 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 shutdown_connections 2024-09-10T22:38:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7c50077f70 0x7f7c5007a430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c6415be00 0x7f7c640c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7c64155460 0x7f7c641f52d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 --2- 172.21.15.66:0/912673843 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7c64070030 0x7f7c64155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 >> 172.21.15.66:0/912673843 conn(0x7f7c640b1950 msgr2=0x7f7c640cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 shutdown_connections 2024-09-10T22:38:26.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.842+0000 7f7c6909b640 1 -- 172.21.15.66:0/912673843 wait complete. 2024-09-10T22:38:26.855 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:38:26.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.958+0000 7f80a3857640 1 -- 172.21.15.66:0/679550626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 msgr2=0x7f809c0d43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.958+0000 7f80a3857640 1 --2- 172.21.15.66:0/679550626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c0d43e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f809c161b30 tx=0x7f809002aa30 comp rx=0 tx=0).stop 2024-09-10T22:38:26.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.958+0000 7f80a3857640 1 -- 172.21.15.66:0/679550626 shutdown_connections 2024-09-10T22:38:26.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.958+0000 7f80a3857640 1 --2- 172.21.15.66:0/679550626 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f809c0d2ce0 0x7f809c0b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.958+0000 7f80a3857640 1 --2- 172.21.15.66:0/679550626 >> v2:172.21.15.94:3300/0 conn(0x7f809c15e2b0 0x7f809c0d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.958+0000 7f80a3857640 1 --2- 172.21.15.66:0/679550626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c0d43e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.958+0000 7f80a3857640 1 -- 172.21.15.66:0/679550626 >> 172.21.15.66:0/679550626 conn(0x7f809c074c20 msgr2=0x7f809c09d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:26.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.959+0000 7f80a3857640 1 -- 172.21.15.66:0/679550626 shutdown_connections 2024-09-10T22:38:26.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.959+0000 7f80a3857640 1 -- 172.21.15.66:0/679550626 wait complete. 2024-09-10T22:38:26.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.960+0000 7f80a3857640 1 Processor -- start 2024-09-10T22:38:26.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.960+0000 7f80a3857640 1 -- start start 2024-09-10T22:38:26.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.960+0000 7f80a3857640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f809c0d2ce0 0x7f809c09dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.960+0000 7f80a3857640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c09f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.960+0000 7f80a3857640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f809c15e2b0 0x7f809c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.960+0000 7f80a3857640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0b4b10 con 0x7f809c0d4000 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a3857640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0b4990 con 0x7f809c15e2b0 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a3857640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f809c0b4c90 con 0x7f809c0d2ce0 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c09f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c09f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34412/0 (socket says 172.21.15.66:34412) 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 -- 172.21.15.66:0/508929776 learned_addr learned my addr 172.21.15.66:0/508929776 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a15f3640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f809c0d2ce0 0x7f809c09dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a1df4640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f809c15e2b0 0x7f809c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 -- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f809c0d2ce0 msgr2=0x7f809c09dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f809c0d2ce0 0x7f809c09dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 -- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f809c15e2b0 msgr2=0x7f809c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f809c15e2b0 0x7f809c154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 -- 172.21.15.66:0/508929776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c154ac0 con 0x7f809c0d4000 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a0df2640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c09f900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f809800d740 tx=0x7f809800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f808dffb640 1 -- 172.21.15.66:0/508929776 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098017070 con 0x7f809c0d4000 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f808dffb640 1 -- 172.21.15.66:0/508929776 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8098009d20 con 0x7f809c0d4000 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f808dffb640 1 -- 172.21.15.66:0/508929776 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80980106c0 con 0x7f809c0d4000 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c151bd0 con 0x7f809c0d4000 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.961+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c152190 con 0x7f809c0d4000 2024-09-10T22:38:26.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.962+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809c0d43e0 con 0x7f809c0d4000 2024-09-10T22:38:26.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.963+0000 7f808dffb640 1 -- 172.21.15.66:0/508929776 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8098010860 con 0x7f809c0d4000 2024-09-10T22:38:26.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.963+0000 7f808dffb640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8080077f70 0x7f808007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:26.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.964+0000 7f808dffb640 1 -- 172.21.15.66:0/508929776 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f809809a140 con 0x7f809c0d4000 2024-09-10T22:38:26.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.964+0000 7f80a15f3640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8080077f70 0x7f808007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:26.967 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.964+0000 7f80a15f3640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8080077f70 0x7f808007a430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8090035000 tx=0x7f809003a040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:26.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:26.968+0000 7f808dffb640 1 -- 172.21.15.66:0/508929776 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f809c154ac0 con 0x7f809c0d4000 2024-09-10T22:38:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.155+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f809c0d29c0 con 0x7f809c0d4000 2024-09-10T22:38:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.156+0000 7f808dffb640 1 -- 172.21.15.66:0/508929776 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16834 (secure 0 0 0) 0x7f8098062c70 con 0x7f809c0d4000 2024-09-10T22:38:27.158 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:27.158 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":36,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:52.690514+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:42.014848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:42.966140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1970715211":"2024-09-11T22:35:52.685163+0000","172.21.15.66:6833/1970715211":"2024-09-11T22:35:52.685163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:38:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8080077f70 msgr2=0x7f808007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8080077f70 0x7f808007a430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8090035000 tx=0x7f809003a040 comp rx=0 tx=0).stop 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 msgr2=0x7f809c09f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c09f900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f809800d740 tx=0x7f809800dc00 comp rx=0 tx=0).stop 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 shutdown_connections 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8080077f70 0x7f808007a430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f809c15e2b0 0x7f809c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f809c0d4000 0x7f809c09f900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 --2- 172.21.15.66:0/508929776 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f809c0d2ce0 0x7f809c09dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 >> 172.21.15.66:0/508929776 conn(0x7f809c074c20 msgr2=0x7f809c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:27.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 shutdown_connections 2024-09-10T22:38:27.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.158+0000 7f80a3857640 1 -- 172.21.15.66:0/508929776 wait complete. 2024-09-10T22:38:27.171 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:38:27.172 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 -- 172.21.15.66:0/2093562229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c161510 msgr2=0x7feb1c0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 --2- 172.21.15.66:0/2093562229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c161510 0x7feb1c0cf6c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feb18005fd0 tx=0x7feb1802a530 comp rx=0 tx=0).stop 2024-09-10T22:38:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 -- 172.21.15.66:0/2093562229 shutdown_connections 2024-09-10T22:38:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 --2- 172.21.15.66:0/2093562229 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb1c0d4b00 0x7feb1c0d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 --2- 172.21.15.66:0/2093562229 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c161510 0x7feb1c0cf6c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 -- 172.21.15.66:0/2093562229 >> 172.21.15.66:0/2093562229 conn(0x7feb1c0d4010 msgr2=0x7feb1c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 -- 172.21.15.66:0/2093562229 shutdown_connections 2024-09-10T22:38:27.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.275+0000 7feb2259f640 1 -- 172.21.15.66:0/2093562229 wait complete. 2024-09-10T22:38:27.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.276+0000 7feb2259f640 1 Processor -- start 2024-09-10T22:38:27.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.276+0000 7feb2259f640 1 -- start start 2024-09-10T22:38:27.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb2259f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb1c0d0750 0x7feb1c1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb2259f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c0d4b00 0x7feb1c1f9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb2259f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb1c161510 0x7feb1c14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb2259f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0c83f0 con 0x7feb1c0d4b00 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb2259f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0c8270 con 0x7feb1c0d0750 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb2259f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7feb1c0c8570 con 0x7feb1c161510 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c0d4b00 0x7feb1c1f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c0d4b00 0x7feb1c1f9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34436/0 (socket says 172.21.15.66:34436) 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 -- 172.21.15.66:0/2244689510 learned_addr learned my addr 172.21.15.66:0/2244689510 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb2159d640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb1c0d0750 0x7feb1c1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb21d9e640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb1c161510 0x7feb1c14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 -- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb1c161510 msgr2=0x7feb1c14bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb1c161510 0x7feb1c14bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 -- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb1c0d0750 msgr2=0x7feb1c1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb1c0d0750 0x7feb1c1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 -- 172.21.15.66:0/2244689510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c1f9870 con 0x7feb1c0d4b00 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.277+0000 7feb20d9c640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c0d4b00 0x7feb1c1f9330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00c920 tx=0x7feb0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.278+0000 7feb11ffb640 1 -- 172.21.15.66:0/2244689510 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c007c70 con 0x7feb1c0d4b00 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.278+0000 7feb11ffb640 1 -- 172.21.15.66:0/2244689510 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb0c00f450 con 0x7feb1c0d4b00 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.278+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c14dde0 con 0x7feb1c0d4b00 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.278+0000 7feb11ffb640 1 -- 172.21.15.66:0/2244689510 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00f8a0 con 0x7feb1c0d4b00 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.278+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c14e3a0 con 0x7feb1c0d4b00 2024-09-10T22:38:27.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.279+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1c0d2a40 con 0x7feb1c0d4b00 2024-09-10T22:38:27.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.279+0000 7feb11ffb640 1 -- 172.21.15.66:0/2244689510 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7feb0c020030 con 0x7feb1c0d4b00 2024-09-10T22:38:27.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.280+0000 7feb11ffb640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7feb00077f70 0x7feb0007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.280+0000 7feb11ffb640 1 -- 172.21.15.66:0/2244689510 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7feb0c09a2b0 con 0x7feb1c0d4b00 2024-09-10T22:38:27.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.280+0000 7feb2159d640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7feb00077f70 0x7feb0007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.280+0000 7feb2159d640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7feb00077f70 0x7feb0007a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feb18033000 tx=0x7feb18004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:27.286 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.284+0000 7feb11ffb640 1 -- 172.21.15.66:0/2244689510 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7feb0c014030 con 0x7feb1c0d4b00 2024-09-10T22:38:27.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.506+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb1c160ba0 con 0x7feb1c0d4b00 2024-09-10T22:38:27.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.506+0000 7feb11ffb640 1 -- 172.21.15.66:0/2244689510 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5215 (secure 0 0 0) 0x7feb0c062de0 con 0x7feb1c0d4b00 2024-09-10T22:38:27.509 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:27.509 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"btime":"2024-09-10T22:35:56:704166+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4252749143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4252749143},{"type":"v1","addr":"172.21.15.94:6835","nonce":4252749143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/58860958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":58860958},{"type":"v1","addr":"172.21.15.94:6837","nonce":58860958}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3000446890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3000446890},{"type":"v1","addr":"172.21.15.66:6835","nonce":3000446890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:56.704154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/956430443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":956430443},{"type":"v1","addr":"172.21.15.66:6834","nonce":956430443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5106,"qdb_cluster":[5106]},"id":2}]} 2024-09-10T22:38:27.509 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7feb00077f70 msgr2=0x7feb0007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7feb00077f70 0x7feb0007a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feb18033000 tx=0x7feb18004730 comp rx=0 tx=0).stop 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c0d4b00 msgr2=0x7feb1c1f9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c0d4b00 0x7feb1c1f9330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00c920 tx=0x7feb0c00cde0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 shutdown_connections 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7feb00077f70 0x7feb0007a430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7feb1c161510 0x7feb1c14bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb1c0d4b00 0x7feb1c1f9330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.508+0000 7feb2259f640 1 --2- 172.21.15.66:0/2244689510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb1c0d0750 0x7feb1c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.509+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 >> 172.21.15.66:0/2244689510 conn(0x7feb1c0d4010 msgr2=0x7feb1c06e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:27.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.509+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 shutdown_connections 2024-09-10T22:38:27.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.509+0000 7feb2259f640 1 -- 172.21.15.66:0/2244689510 wait complete. 2024-09-10T22:38:27.522 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:27.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 -- 172.21.15.66:0/3477229404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015db50 msgr2=0x7f9db015ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 --2- 172.21.15.66:0/3477229404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015db50 0x7f9db015ea70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9db00cf1b0 tx=0x7f9dac02a4d0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.627 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 -- 172.21.15.66:0/3477229404 shutdown_connections 2024-09-10T22:38:27.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 --2- 172.21.15.66:0/3477229404 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9db00ce0f0 0x7f9db00c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 --2- 172.21.15.66:0/3477229404 >> v2:172.21.15.94:3300/0 conn(0x7f9db00c62f0 0x7f9db00cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 --2- 172.21.15.66:0/3477229404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db015db50 0x7f9db015ea70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 -- 172.21.15.66:0/3477229404 >> 172.21.15.66:0/3477229404 conn(0x7f9db00b1950 msgr2=0x7f9db00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:27.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 -- 172.21.15.66:0/3477229404 shutdown_connections 2024-09-10T22:38:27.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.625+0000 7f9db820f640 1 -- 172.21.15.66:0/3477229404 wait complete. 2024-09-10T22:38:27.628 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.626+0000 7f9db820f640 1 Processor -- start 2024-09-10T22:38:27.629 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.626+0000 7f9db820f640 1 -- start start 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db820f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9db00c62f0 0x7f9db01e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db820f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db00ce0f0 0x7f9db01f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db820f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db015db50 0x7f9db01e6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db820f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00cc7c0 con 0x7f9db00ce0f0 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db820f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00cc940 con 0x7f9db015db50 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db820f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9db00ccac0 con 0x7f9db00c62f0 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db57aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db00ce0f0 0x7f9db01f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db57aa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db00ce0f0 0x7f9db01f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34460/0 (socket says 172.21.15.66:34460) 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db57aa640 1 -- 172.21.15.66:0/2385039975 learned_addr learned my addr 172.21.15.66:0/2385039975 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db67ac640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db015db50 0x7f9db01e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.627+0000 7f9db5fab640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9db00c62f0 0x7f9db01e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.631 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db57aa640 1 -- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9db00c62f0 msgr2=0x7f9db01e3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db57aa640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9db00c62f0 0x7f9db01e3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db57aa640 1 -- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db015db50 msgr2=0x7f9db01e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db57aa640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db015db50 0x7f9db01e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db57aa640 1 -- 172.21.15.66:0/2385039975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db00b5760 con 0x7f9db00ce0f0 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db57aa640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db00ce0f0 0x7f9db01f86e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9da000ea20 tx=0x7f9da000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9da67fc640 1 -- 172.21.15.66:0/2385039975 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000cd40 con 0x7f9db00ce0f0 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9da67fc640 1 -- 172.21.15.66:0/2385039975 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9da0016410 con 0x7f9db00ce0f0 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db00c7c70 con 0x7f9db00ce0f0 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9da67fc640 1 -- 172.21.15.66:0/2385039975 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0016860 con 0x7f9db00ce0f0 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.628+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db01f9130 con 0x7f9db00ce0f0 2024-09-10T22:38:27.632 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.629+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db00cdbb0 con 0x7f9db00ce0f0 2024-09-10T22:38:27.633 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.630+0000 7f9da67fc640 1 -- 172.21.15.66:0/2385039975 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9da000b040 con 0x7f9db00ce0f0 2024-09-10T22:38:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.630+0000 7f9da67fc640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d94077f70 0x7f9d9407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.630+0000 7f9da67fc640 1 -- 172.21.15.66:0/2385039975 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f9da0099ff0 con 0x7f9db00ce0f0 2024-09-10T22:38:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.630+0000 7f9db5fab640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d94077f70 0x7f9d9407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.634 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.631+0000 7f9db5fab640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d94077f70 0x7f9d9407a430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9dac033000 tx=0x7f9dac004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:27.636 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.634+0000 7f9da67fc640 1 -- 172.21.15.66:0/2385039975 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9db00c7c70 con 0x7f9db00ce0f0 2024-09-10T22:38:27.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.855+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9db00c4e70 con 0x7f9db00ce0f0 2024-09-10T22:38:27.857 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.855+0000 7f9da67fc640 1 -- 172.21.15.66:0/2385039975 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5215 (secure 0 0 0) 0x7f9da0062b20 con 0x7f9db00ce0f0 2024-09-10T22:38:27.858 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:27.858 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":24,"btime":"2024-09-10T22:35:56:704166+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4252749143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4252749143},{"type":"v1","addr":"172.21.15.94:6835","nonce":4252749143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/58860958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":58860958},{"type":"v1","addr":"172.21.15.94:6837","nonce":58860958}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3000446890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3000446890},{"type":"v1","addr":"172.21.15.66:6835","nonce":3000446890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:35:44.016061+0000","modified":"2024-09-10T22:35:56.704154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/956430443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":956430443},{"type":"v1","addr":"172.21.15.66:6834","nonce":956430443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5106,"qdb_cluster":[5106]},"id":2}]} 2024-09-10T22:38:27.858 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 24 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d94077f70 msgr2=0x7f9d9407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d94077f70 0x7f9d9407a430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9dac033000 tx=0x7f9dac004730 comp rx=0 tx=0).stop 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db00ce0f0 msgr2=0x7f9db01f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db00ce0f0 0x7f9db01f86e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9da000ea20 tx=0x7f9da000eee0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 shutdown_connections 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9d94077f70 0x7f9d9407a430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db015db50 0x7f9db01e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.860 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9db00ce0f0 0x7f9db01f86e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 --2- 172.21.15.66:0/2385039975 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9db00c62f0 0x7f9db01e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 >> 172.21.15.66:0/2385039975 conn(0x7f9db00b1950 msgr2=0x7f9db00ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 shutdown_connections 2024-09-10T22:38:27.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.858+0000 7f9db820f640 1 -- 172.21.15.66:0/2385039975 wait complete. 2024-09-10T22:38:27.870 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:38:27.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 -- 172.21.15.66:0/132293342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 msgr2=0x7f596c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 --2- 172.21.15.66:0/132293342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 0x7f596c0cdbb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5954009830 tx=0x7f595402a470 comp rx=0 tx=0).stop 2024-09-10T22:38:27.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 -- 172.21.15.66:0/132293342 shutdown_connections 2024-09-10T22:38:27.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 --2- 172.21.15.66:0/132293342 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 0x7f596c0cdbb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 --2- 172.21.15.66:0/132293342 >> v2:172.21.15.94:3300/0 conn(0x7f596c15db50 0x7f596c15ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 -- 172.21.15.66:0/132293342 >> 172.21.15.66:0/132293342 conn(0x7f596c0b1950 msgr2=0x7f596c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 -- 172.21.15.66:0/132293342 shutdown_connections 2024-09-10T22:38:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.973+0000 7f597167f640 1 -- 172.21.15.66:0/132293342 wait complete. 2024-09-10T22:38:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.975+0000 7f597167f640 1 Processor -- start 2024-09-10T22:38:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.975+0000 7f597167f640 1 -- start start 2024-09-10T22:38:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.975+0000 7f597167f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 0x7f596c1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.975+0000 7f597167f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c0ce0f0 0x7f596c148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.975+0000 7f597167f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c15db50 0x7f596c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f597167f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f596c0c57a0 con 0x7f596c0ce0f0 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f597167f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f596c0c5920 con 0x7f596c15db50 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f597167f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f596c0c5aa0 con 0x7f596c0c62f0 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c0ce0f0 0x7f596c148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c0ce0f0 0x7f596c148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34472/0 (socket says 172.21.15.66:34472) 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 -- 172.21.15.66:0/3758342283 learned_addr learned my addr 172.21.15.66:0/3758342283 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596b7fe640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c15db50 0x7f596c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596affd640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 0x7f596c1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 -- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 msgr2=0x7f596c1f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 0x7f596c1f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 -- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c15db50 msgr2=0x7f596c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:27.981 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c15db50 0x7f596c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 -- 172.21.15.66:0/3758342283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1f6760 con 0x7f596c0ce0f0 2024-09-10T22:38:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f596a7fc640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c0ce0f0 0x7f596c148850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5954000c00 tx=0x7f5954032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f59637fe640 1 -- 172.21.15.66:0/3758342283 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954040070 con 0x7f596c0ce0f0 2024-09-10T22:38:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f59637fe640 1 -- 172.21.15.66:0/3758342283 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59540038e0 con 0x7f596c0ce0f0 2024-09-10T22:38:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1f6a40 con 0x7f596c0ce0f0 2024-09-10T22:38:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f59637fe640 1 -- 172.21.15.66:0/3758342283 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954003d30 con 0x7f596c0ce0f0 2024-09-10T22:38:27.982 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.976+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1f7000 con 0x7f596c0ce0f0 2024-09-10T22:38:27.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.978+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c0cdbb0 con 0x7f596c0ce0f0 2024-09-10T22:38:27.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.978+0000 7f59637fe640 1 -- 172.21.15.66:0/3758342283 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f595404d030 con 0x7f596c0ce0f0 2024-09-10T22:38:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.979+0000 7f59637fe640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f594c077f70 0x7f594c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.979+0000 7f59637fe640 1 -- 172.21.15.66:0/3758342283 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f59540bb120 con 0x7f596c0ce0f0 2024-09-10T22:38:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.979+0000 7f596affd640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f594c077f70 0x7f594c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:27.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.979+0000 7f596affd640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f594c077f70 0x7f594c07a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59640099a0 tx=0x7f5964009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:27.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:27.983+0000 7f59637fe640 1 -- 172.21.15.66:0/3758342283 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5954083570 con 0x7f596c0ce0f0 2024-09-10T22:38:28.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.171+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f596c0c4e70 con 0x7f596c0ce0f0 2024-09-10T22:38:28.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.172+0000 7f59637fe640 1 -- 172.21.15.66:0/3758342283 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16834 (secure 0 0 0) 0x7f5954087fa0 con 0x7f596c0ce0f0 2024-09-10T22:38:28.174 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:28.174 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":36,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:35:52.690514+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:35:42.014848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T22:35:42.966140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/1970715211":"2024-09-11T22:35:52.685163+0000","172.21.15.66:6833/1970715211":"2024-09-11T22:35:52.685163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:38:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f594c077f70 msgr2=0x7f594c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f594c077f70 0x7f594c07a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59640099a0 tx=0x7f5964009500 comp rx=0 tx=0).stop 2024-09-10T22:38:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c0ce0f0 msgr2=0x7f596c148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.176 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c0ce0f0 0x7f596c148850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5954000c00 tx=0x7f5954032c10 comp rx=0 tx=0).stop 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 shutdown_connections 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f594c077f70 0x7f594c07a430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f596c15db50 0x7f596c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f596c0ce0f0 0x7f596c148850 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 --2- 172.21.15.66:0/3758342283 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f596c0c62f0 0x7f596c1f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 >> 172.21.15.66:0/3758342283 conn(0x7f596c0b1950 msgr2=0x7f596c0cc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 shutdown_connections 2024-09-10T22:38:28.177 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.174+0000 7f597167f640 1 -- 172.21.15.66:0/3758342283 wait complete. 2024-09-10T22:38:28.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:38:28.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.291+0000 7ff3d1c41640 1 -- 172.21.15.66:0/709954316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 msgr2=0x7ff3cc15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.291+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/709954316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc15c0f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009830 tx=0x7ff3bc02a510 comp rx=0 tx=0).stop 2024-09-10T22:38:28.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.291+0000 7ff3d1c41640 1 -- 172.21.15.66:0/709954316 shutdown_connections 2024-09-10T22:38:28.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.291+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/709954316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc15c0f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.291+0000 7ff3d1c41640 1 -- 172.21.15.66:0/709954316 >> 172.21.15.66:0/709954316 conn(0x7ff3cc0cfd00 msgr2=0x7ff3cc0d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.291+0000 7ff3d1c41640 1 -- 172.21.15.66:0/709954316 shutdown_connections 2024-09-10T22:38:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.291+0000 7ff3d1c41640 1 -- 172.21.15.66:0/709954316 wait complete. 2024-09-10T22:38:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.292+0000 7ff3d1c41640 1 Processor -- start 2024-09-10T22:38:28.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.292+0000 7ff3d1c41640 1 -- start start 2024-09-10T22:38:28.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3d1c41640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3d1c41640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3cc0746f0 0x7ff3cc14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3d1c41640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc0cee90 0x7ff3cc12dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3d1c41640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc167330 con 0x7ff3cc072a80 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3d1c41640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1671b0 con 0x7ff3cc0cee90 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3d1c41640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc1674b0 con 0x7ff3cc0746f0 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34498/0 (socket says 172.21.15.66:34498) 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.293+0000 7ff3cb7fe640 1 -- 172.21.15.66:0/801603881 learned_addr learned my addr 172.21.15.66:0/801603881 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3caffd640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3cc0746f0 0x7ff3cc14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cbfff640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc0cee90 0x7ff3cc12dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cb7fe640 1 -- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3cc0746f0 msgr2=0x7ff3cc14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cb7fe640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3cc0746f0 0x7ff3cc14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cb7fe640 1 -- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc0cee90 msgr2=0x7ff3cc12dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cb7fe640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc0cee90 0x7ff3cc12dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cb7fe640 1 -- 172.21.15.66:0/801603881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc151180 con 0x7ff3cc072a80 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cbfff640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc0cee90 0x7ff3cc12dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3caffd640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3cc0746f0 0x7ff3cc14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3cb7fe640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc150620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009950 tx=0x7ff3bc032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3b3fff640 1 -- 172.21.15.66:0/801603881 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc040070 con 0x7ff3cc072a80 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3b3fff640 1 -- 172.21.15.66:0/801603881 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3bc005310 con 0x7ff3cc072a80 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc151460 con 0x7ff3cc072a80 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3b3fff640 1 -- 172.21.15.66:0/801603881 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0047a0 con 0x7ff3cc072a80 2024-09-10T22:38:28.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.294+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc150b60 con 0x7ff3cc072a80 2024-09-10T22:38:28.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.295+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc15c0f0 con 0x7ff3cc072a80 2024-09-10T22:38:28.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.296+0000 7ff3b3fff640 1 -- 172.21.15.66:0/801603881 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff3bc04c030 con 0x7ff3cc072a80 2024-09-10T22:38:28.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.296+0000 7ff3b3fff640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3b8077f70 0x7ff3b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.296+0000 7ff3b3fff640 1 -- 172.21.15.66:0/801603881 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7ff3bc0bb300 con 0x7ff3cc072a80 2024-09-10T22:38:28.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.296+0000 7ff3caffd640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3b8077f70 0x7ff3b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.297+0000 7ff3caffd640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3b8077f70 0x7ff3b807a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0af670 tx=0x7ff3c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:28.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.301+0000 7ff3b3fff640 1 -- 172.21.15.66:0/801603881 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff3bc083d10 con 0x7ff3cc072a80 2024-09-10T22:38:28.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.522+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff3cc159fa0 con 0x7ff3cc072a80 2024-09-10T22:38:28.525 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:38:28.523+0000 7fe21e4b6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:38:28.526 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.524+0000 7ff3b3fff640 1 -- 172.21.15.66:0/801603881 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7ff3bc083610 con 0x7ff3cc072a80 2024-09-10T22:38:28.526 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.527+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3b8077f70 msgr2=0x7ff3b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3b8077f70 0x7ff3b807a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc0af670 tx=0x7ff3c0009040 comp rx=0 tx=0).stop 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 msgr2=0x7ff3cc150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc150620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009950 tx=0x7ff3bc032c10 comp rx=0 tx=0).stop 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 shutdown_connections 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff3b8077f70 0x7ff3b807a430 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3cc0cee90 0x7ff3cc12dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff3cc0746f0 0x7ff3cc14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 --2- 172.21.15.66:0/801603881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff3cc072a80 0x7ff3cc150620 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 >> 172.21.15.66:0/801603881 conn(0x7ff3cc0cfd00 msgr2=0x7ff3cc157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:28.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 shutdown_connections 2024-09-10T22:38:28.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.528+0000 7ff3d1c41640 1 -- 172.21.15.66:0/801603881 wait complete. 2024-09-10T22:38:28.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:38:28.568 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:38:28.650 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 -- 172.21.15.66:0/1596432317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc0d1de0 msgr2=0x7f86bc162830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 --2- 172.21.15.66:0/1596432317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc0d1de0 0x7f86bc162830 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0b6e00 tx=0x7f86a402a5f0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 -- 172.21.15.66:0/1596432317 shutdown_connections 2024-09-10T22:38:28.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 --2- 172.21.15.66:0/1596432317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc0d1de0 0x7f86bc162830 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 --2- 172.21.15.66:0/1596432317 >> v2:172.21.15.94:3300/0 conn(0x7f86bc15e390 0x7f86bc0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 -- 172.21.15.66:0/1596432317 >> 172.21.15.66:0/1596432317 conn(0x7f86bc0d4210 msgr2=0x7f86bc0d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:28.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 -- 172.21.15.66:0/1596432317 shutdown_connections 2024-09-10T22:38:28.651 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.648+0000 7f86c0d52640 1 -- 172.21.15.66:0/1596432317 wait complete. 2024-09-10T22:38:28.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.649+0000 7f86c0d52640 1 Processor -- start 2024-09-10T22:38:28.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.649+0000 7f86c0d52640 1 -- start start 2024-09-10T22:38:28.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86c0d52640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86bc0d25c0 0x7f86bc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86c0d52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc15e390 0x7f86bc160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.653 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86c0d52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1614a0 0x7f86bc09df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86c0d52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc0d2bd0 con 0x7f86bc15e390 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86c0d52640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc0d2a50 con 0x7f86bc1614a0 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86c0d52640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc0d2d50 con 0x7f86bc0d25c0 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc15e390 0x7f86bc160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc15e390 0x7f86bc160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34516/0 (socket says 172.21.15.66:34516) 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 -- 172.21.15.66:0/915857179 learned_addr learned my addr 172.21.15.66:0/915857179 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86ba59c640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86bc0d25c0 0x7f86bc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86bad9d640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1614a0 0x7f86bc09df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 -- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86bc0d25c0 msgr2=0x7f86bc15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86bc0d25c0 0x7f86bc15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 -- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1614a0 msgr2=0x7f86bc09df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1614a0 0x7f86bc09df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:28.654 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86b3fff640 1 -- 172.21.15.66:0/915857179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0a1920 con 0x7f86bc15e390 2024-09-10T22:38:28.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.650+0000 7f86bad9d640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1614a0 0x7f86bc09df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:28.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.651+0000 7f86b3fff640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc15e390 0x7f86bc160530 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f86a4034040 tx=0x7f86a4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:28.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.651+0000 7f86b2ffd640 1 -- 172.21.15.66:0/915857179 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a403e030 con 0x7f86bc15e390 2024-09-10T22:38:28.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.651+0000 7f86b2ffd640 1 -- 172.21.15.66:0/915857179 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a4003e00 con 0x7f86bc15e390 2024-09-10T22:38:28.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.651+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc0a1c00 con 0x7f86bc15e390 2024-09-10T22:38:28.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.651+0000 7f86b2ffd640 1 -- 172.21.15.66:0/915857179 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4035040 con 0x7f86bc15e390 2024-09-10T22:38:28.655 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.651+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc15bbf0 con 0x7f86bc15e390 2024-09-10T22:38:28.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.652+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc0d45f0 con 0x7f86bc15e390 2024-09-10T22:38:28.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.652+0000 7f86b2ffd640 1 -- 172.21.15.66:0/915857179 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f86a40049c0 con 0x7f86bc15e390 2024-09-10T22:38:28.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.653+0000 7f86b2ffd640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f869c077f70 0x7f869c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:28.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.653+0000 7f86b2ffd640 1 -- 172.21.15.66:0/915857179 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5881+0+0 (secure 0 0 0) 0x7f86a40ba780 con 0x7f86bc15e390 2024-09-10T22:38:28.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.653+0000 7f86ba59c640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f869c077f70 0x7f869c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:28.656 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.654+0000 7f86ba59c640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f869c077f70 0x7f869c07a430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0b5280 tx=0x7f86ac00b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:28.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.657+0000 7f86b2ffd640 1 -- 172.21.15.66:0/915857179 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f86bc0d45f0 con 0x7f86bc15e390 2024-09-10T22:38:28.883 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:28.881+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f86bc162830 con 0x7f86bc15e390 2024-09-10T22:38:29.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.529+0000 7f86b2ffd640 1 -- 172.21.15.66:0/915857179 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f86a4082b80 con 0x7f86bc15e390 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.531+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f869c077f70 msgr2=0x7f869c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.531+0000 7f86c0d52640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f869c077f70 0x7f869c07a430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0b5280 tx=0x7f86ac00b390 comp rx=0 tx=0).stop 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.531+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc15e390 msgr2=0x7f86bc160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.531+0000 7f86c0d52640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc15e390 0x7f86bc160530 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f86a4034040 tx=0x7f86a4003910 comp rx=0 tx=0).stop 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 shutdown_connections 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f869c077f70 0x7f869c07a430 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86bc1614a0 0x7f86bc09df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f86bc15e390 0x7f86bc160530 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 --2- 172.21.15.66:0/915857179 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f86bc0d25c0 0x7f86bc15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 >> 172.21.15.66:0/915857179 conn(0x7f86bc0d4210 msgr2=0x7f86bc0b9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:29.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 shutdown_connections 2024-09-10T22:38:29.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.532+0000 7f86c0d52640 1 -- 172.21.15.66:0/915857179 wait complete. 2024-09-10T22:38:29.548 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:38:29.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 -- 172.21.15.66:0/869488175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 msgr2=0x7f7f4415b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/869488175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4415b5a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f440b3ef0 tx=0x7f7f3402a980 comp rx=0 tx=0).stop 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 -- 172.21.15.66:0/869488175 shutdown_connections 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/869488175 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f440ce1d0 0x7f7f4415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/869488175 >> v2:172.21.15.94:3300/0 conn(0x7f7f440cd570 0x7f7f440cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/869488175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4415b5a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 -- 172.21.15.66:0/869488175 >> 172.21.15.66:0/869488175 conn(0x7f7f440b19c0 msgr2=0x7f7f440b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 -- 172.21.15.66:0/869488175 shutdown_connections 2024-09-10T22:38:29.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.702+0000 7f7f49cd4640 1 -- 172.21.15.66:0/869488175 wait complete. 2024-09-10T22:38:29.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.703+0000 7f7f49cd4640 1 Processor -- start 2024-09-10T22:38:29.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.703+0000 7f7f49cd4640 1 -- start start 2024-09-10T22:38:29.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f49cd4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f440cd570 0x7f7f4414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:29.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f49cd4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f440ce1d0 0x7f7f4409bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f49cd4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4409c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f49cd4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4415f060 con 0x7f7f4415b1c0 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f49cd4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4415eee0 con 0x7f7f440ce1d0 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f49cd4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7f440cc810 con 0x7f7f440cd570 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4409c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4409c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34528/0 (socket says 172.21.15.66:34528) 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f43fff640 1 -- 172.21.15.66:0/2896591380 learned_addr learned my addr 172.21.15.66:0/2896591380 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f437fe640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f440cd570 0x7f7f4414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.704+0000 7f7f42ffd640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f440ce1d0 0x7f7f4409bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f43fff640 1 -- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f440cd570 msgr2=0x7f7f4414dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f43fff640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f440cd570 0x7f7f4414dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f43fff640 1 -- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f440ce1d0 msgr2=0x7f7f4409bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:29.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f43fff640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f440ce1d0 0x7f7f4409bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:29.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f43fff640 1 -- 172.21.15.66:0/2896591380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4406f630 con 0x7f7f4415b1c0 2024-09-10T22:38:29.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f43fff640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4409c440 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c750 tx=0x7f7f3c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:29.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f2bfff640 1 -- 172.21.15.66:0/2896591380 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00ce60 con 0x7f7f4415b1c0 2024-09-10T22:38:29.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f2bfff640 1 -- 172.21.15.66:0/2896591380 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f3c007ca0 con 0x7f7f4415b1c0 2024-09-10T22:38:29.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4406f910 con 0x7f7f4415b1c0 2024-09-10T22:38:29.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f2bfff640 1 -- 172.21.15.66:0/2896591380 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c005280 con 0x7f7f4415b1c0 2024-09-10T22:38:29.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.705+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4406fed0 con 0x7f7f4415b1c0 2024-09-10T22:38:29.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.706+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f440ccd40 con 0x7f7f4415b1c0 2024-09-10T22:38:29.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.707+0000 7f7f2bfff640 1 -- 172.21.15.66:0/2896591380 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7f3c005420 con 0x7f7f4415b1c0 2024-09-10T22:38:29.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.707+0000 7f7f2bfff640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7f30077fa0 0x7f7f3007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:29.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.708+0000 7f7f2bfff640 1 -- 172.21.15.66:0/2896591380 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5881+0+0 (secure 0 0 0) 0x7f7f3c09a0c0 con 0x7f7f4415b1c0 2024-09-10T22:38:29.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.708+0000 7f7f437fe640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7f30077fa0 0x7f7f3007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:29.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.708+0000 7f7f437fe640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7f30077fa0 0x7f7f3007a460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f34035000 tx=0x7f7f34004ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:29.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.712+0000 7f7f2bfff640 1 -- 172.21.15.66:0/2896591380 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7f3c014030 con 0x7f7f4415b1c0 2024-09-10T22:38:29.899 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:29.897+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f44152ec0 con 0x7f7f4415b1c0 2024-09-10T22:38:30.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.531+0000 7f7f2bfff640 1 -- 172.21.15.66:0/2896591380 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7f7f3c06b2c0 con 0x7f7f4415b1c0 2024-09-10T22:38:30.533 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:38:30.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7f30077fa0 msgr2=0x7f7f3007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:30.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7f30077fa0 0x7f7f3007a460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f34035000 tx=0x7f7f34004ab0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 msgr2=0x7f7f4409c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:30.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4409c440 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c750 tx=0x7f7f3c00cc10 comp rx=0 tx=0).stop 2024-09-10T22:38:30.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 shutdown_connections 2024-09-10T22:38:30.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7f30077fa0 0x7f7f3007a460 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f4415b1c0 0x7f7f4409c440 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f440ce1d0 0x7f7f4409bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 --2- 172.21.15.66:0/2896591380 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f440cd570 0x7f7f4414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 >> 172.21.15.66:0/2896591380 conn(0x7f7f440b19c0 msgr2=0x7f7f440b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 shutdown_connections 2024-09-10T22:38:30.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.534+0000 7f7f49cd4640 1 -- 172.21.15.66:0/2896591380 wait complete. 2024-09-10T22:38:30.550 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:38:30.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/3064024435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f00d3d90 msgr2=0x7ff1f00d41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:30.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/3064024435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f00d3d90 0x7ff1f00d41d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4005fd0 tx=0x7ff1e402a400 comp rx=0 tx=0).stop 2024-09-10T22:38:30.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/3064024435 shutdown_connections 2024-09-10T22:38:30.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/3064024435 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f00d3d90 0x7ff1f00d41d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/3064024435 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f015e2b0 0x7ff1f00d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/3064024435 >> 172.21.15.66:0/3064024435 conn(0x7ff1f008dac0 msgr2=0x7ff1f009d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/3064024435 shutdown_connections 2024-09-10T22:38:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.662+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/3064024435 wait complete. 2024-09-10T22:38:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.664+0000 7ff1f5ca9640 1 Processor -- start 2024-09-10T22:38:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.664+0000 7ff1f5ca9640 1 -- start start 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1f5ca9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f00d3d90 0x7ff1f00b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1f5ca9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f015e2b0 0x7ff1f00c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1f5ca9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f0161e00 0x7ff1f015a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1f5ca9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00d4ca0 con 0x7ff1f0161e00 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1f5ca9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00d4b20 con 0x7ff1f015e2b0 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1f5ca9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f01629c0 con 0x7ff1f00d3d90 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f0161e00 0x7ff1f015a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f0161e00 0x7ff1f015a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34554/0 (socket says 172.21.15.66:34554) 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 -- 172.21.15.66:0/1906960576 learned_addr learned my addr 172.21.15.66:0/1906960576 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1ef7fe640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f00d3d90 0x7ff1f00b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1eeffd640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f015e2b0 0x7ff1f00c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 -- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f00d3d90 msgr2=0x7ff1f00b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f00d3d90 0x7ff1f00b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 -- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f015e2b0 msgr2=0x7ff1f00c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f015e2b0 0x7ff1f00c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 -- 172.21.15.66:0/1906960576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00b0e70 con 0x7ff1f0161e00 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1effff640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f0161e00 0x7ff1f015a950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b9c0 tx=0x7ff1e800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.665+0000 7ff1d7fff640 1 -- 172.21.15.66:0/1906960576 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8019070 con 0x7ff1f0161e00 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.666+0000 7ff1d7fff640 1 -- 172.21.15.66:0/1906960576 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1e800cbb0 con 0x7ff1f0161e00 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.666+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00b1150 con 0x7ff1f0161e00 2024-09-10T22:38:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.666+0000 7ff1d7fff640 1 -- 172.21.15.66:0/1906960576 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8014440 con 0x7ff1f0161e00 2024-09-10T22:38:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.666+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f00b1710 con 0x7ff1f0161e00 2024-09-10T22:38:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.667+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f00d0040 con 0x7ff1f0161e00 2024-09-10T22:38:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.667+0000 7ff1d7fff640 1 -- 172.21.15.66:0/1906960576 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff1e801d3b0 con 0x7ff1f0161e00 2024-09-10T22:38:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.668+0000 7ff1d7fff640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff1dc077f70 0x7ff1dc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.668+0000 7ff1d7fff640 1 -- 172.21.15.66:0/1906960576 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5443+0+0 (secure 0 0 0) 0x7ff1e8099ac0 con 0x7ff1f0161e00 2024-09-10T22:38:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.668+0000 7ff1ef7fe640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff1dc077f70 0x7ff1dc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:30.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.668+0000 7ff1ef7fe640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff1dc077f70 0x7ff1dc07a430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009990 tx=0x7ff1e0009110 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:30.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.672+0000 7ff1d7fff640 1 -- 172.21.15.66:0/1906960576 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff1e8066b30 con 0x7ff1f0161e00 2024-09-10T22:38:30.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:30.859+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff1f006d440 con 0x7ff1f0161e00 2024-09-10T22:38:31.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.535+0000 7ff1d7fff640 1 -- 172.21.15.66:0/1906960576 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7ff1e8069e20 con 0x7ff1f0161e00 2024-09-10T22:38:31.538 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:38:31.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff1dc077f70 msgr2=0x7ff1dc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:31.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff1dc077f70 0x7ff1dc07a430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009990 tx=0x7ff1e0009110 comp rx=0 tx=0).stop 2024-09-10T22:38:31.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f0161e00 msgr2=0x7ff1f015a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:31.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f0161e00 0x7ff1f015a950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b9c0 tx=0x7ff1e800bdc0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 shutdown_connections 2024-09-10T22:38:31.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff1dc077f70 0x7ff1dc07a430 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009990 tx=0x7ff1e0009110 comp rx=0 tx=0).stop 2024-09-10T22:38:31.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1f0161e00 0x7ff1f015a950 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f015e2b0 0x7ff1f00c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 --2- 172.21.15.66:0/1906960576 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff1f00d3d90 0x7ff1f00b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.539+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 >> 172.21.15.66:0/1906960576 conn(0x7ff1f008dac0 msgr2=0x7ff1f01e7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:31.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.540+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 shutdown_connections 2024-09-10T22:38:31.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.540+0000 7ff1f5ca9640 1 -- 172.21.15.66:0/1906960576 wait complete. 2024-09-10T22:38:31.557 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:38:31.559 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:38:31.606 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-10T22:38:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-09-10T22:38:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:38:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-09-10T22:38:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.606 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:38:31.607 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-09-10T22:38:31.608 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on skipped message = False 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-09-10T22:38:31.609 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-09-10T22:38:31.610 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-09-10T22:38:31.611 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-09-10T22:38:31.612 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 30 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-09-10T22:38:31.613 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:38:31.614 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-09-10T22:38:31.615 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-10T22:38:31.616 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-10T22:38:31.617 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:38:31.618 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on skipped message = False 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:38:31.619 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-10T22:38:31.620 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.621 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-10T22:38:31.622 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:31.623 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 30 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-10T22:38:31.624 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:38:31.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T22:38:31.626 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T22:38:31.627 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T22:38:31.627 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T22:38:31.627 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2024-09-10T22:38:31.628 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1647686475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480d07a0 msgr2=0x7f3f480cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1647686475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480d07a0 0x7f3f480cf680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3f48161b90 tx=0x7f3f3802aa30 comp rx=0 tx=0).stop 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1647686475 shutdown_connections 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1647686475 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f480cfbc0 0x7f3f480d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1647686475 >> v2:172.21.15.94:3300/0 conn(0x7f3f480d2d70 0x7f3f480d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1647686475 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480d07a0 0x7f3f480cf680 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1647686475 >> 172.21.15.66:0/1647686475 conn(0x7f3f480d4060 msgr2=0x7f3f480d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1647686475 shutdown_connections 2024-09-10T22:38:31.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.734+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1647686475 wait complete. 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.735+0000 7f3f4dfb4640 1 Processor -- start 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f4dfb4640 1 -- start start 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f4dfb4640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480cfbc0 0x7f3f480b9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f4dfb4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f480d07a0 0x7f3f4814e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f4dfb4640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f480d2d70 0x7f3f4806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f4dfb4640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480b3d50 con 0x7f3f480cfbc0 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f4dfb4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480b3bd0 con 0x7f3f480d07a0 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f4dfb4640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480b3ed0 con 0x7f3f480d2d70 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480cfbc0 0x7f3f480b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f477fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480cfbc0 0x7f3f480b9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34580/0 (socket says 172.21.15.66:34580) 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.736+0000 7f3f477fe640 1 -- 172.21.15.66:0/1089021216 learned_addr learned my addr 172.21.15.66:0/1089021216 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f47fff640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f480d2d70 0x7f3f4806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:31.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f46ffd640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f480d07a0 0x7f3f4814e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f477fe640 1 -- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f480d2d70 msgr2=0x7f3f4806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f477fe640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f480d2d70 0x7f3f4806d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f477fe640 1 -- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f480d07a0 msgr2=0x7f3f4814e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f477fe640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f480d07a0 0x7f3f4814e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f477fe640 1 -- 172.21.15.66:0/1089021216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f480b83e0 con 0x7f3f480cfbc0 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f46ffd640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f480d07a0 0x7f3f4814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f47fff640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f480d2d70 0x7f3f4806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f477fe640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480cfbc0 0x7f3f480b9750 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009bf0 tx=0x7f3f38003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f3ffff640 1 -- 172.21.15.66:0/1089021216 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38047070 con 0x7f3f480cfbc0 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f3ffff640 1 -- 172.21.15.66:0/1089021216 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f38004250 con 0x7f3f480cfbc0 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f480b86b0 con 0x7f3f480cfbc0 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f3ffff640 1 -- 172.21.15.66:0/1089021216 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f38033050 con 0x7f3f480cfbc0 2024-09-10T22:38:31.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.737+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f480b8f50 con 0x7f3f480cfbc0 2024-09-10T22:38:31.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.738+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f480cf680 con 0x7f3f480cfbc0 2024-09-10T22:38:31.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.739+0000 7f3f3ffff640 1 -- 172.21.15.66:0/1089021216 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3f38004660 con 0x7f3f480cfbc0 2024-09-10T22:38:31.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.739+0000 7f3f3ffff640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3f28077f70 0x7f3f2807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:31.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.739+0000 7f3f3ffff640 1 -- 172.21.15.66:0/1089021216 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f3f380bb220 con 0x7f3f480cfbc0 2024-09-10T22:38:31.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.740+0000 7f3f46ffd640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3f28077f70 0x7f3f2807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:31.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.740+0000 7f3f46ffd640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3f28077f70 0x7f3f2807a430 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f480caf10 tx=0x7f3f30008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:31.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.743+0000 7f3f3ffff640 1 -- 172.21.15.66:0/1089021216 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3f38083fc0 con 0x7f3f480cfbc0 2024-09-10T22:38:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.934+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f3f480d3e40 con 0x7f3f480cfbc0 2024-09-10T22:38:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.941+0000 7f3f3ffff640 1 -- 172.21.15.66:0/1089021216 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v29) ==== 71+0+0 (secure 0 0 0) 0x7f3f380838c0 con 0x7f3f480cfbc0 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3f28077f70 msgr2=0x7f3f2807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3f28077f70 0x7f3f2807a430 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f480caf10 tx=0x7f3f30008040 comp rx=0 tx=0).stop 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480cfbc0 msgr2=0x7f3f480b9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480cfbc0 0x7f3f480b9750 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009bf0 tx=0x7f3f38003f10 comp rx=0 tx=0).stop 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 shutdown_connections 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3f28077f70 0x7f3f2807a430 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f3f480d2d70 0x7f3f4806d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3f480d07a0 0x7f3f4814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 --2- 172.21.15.66:0/1089021216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3f480cfbc0 0x7f3f480b9750 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 >> 172.21.15.66:0/1089021216 conn(0x7f3f480d4060 msgr2=0x7f3f481584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:31.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 shutdown_connections 2024-09-10T22:38:31.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:31.944+0000 7f3f4dfb4640 1 -- 172.21.15.66:0/1089021216 wait complete. 2024-09-10T22:38:31.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-09-10T22:38:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/1688951006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180ca4b0 msgr2=0x7f2e180ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/1688951006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180ca4b0 0x7f2e180ca920 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005fd0 tx=0x7f2e1002a820 comp rx=0 tx=0).stop 2024-09-10T22:38:32.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/1688951006 shutdown_connections 2024-09-10T22:38:32.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/1688951006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e180ca4b0 0x7f2e180ca920 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/1688951006 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e18152a40 0x7f2e18152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/1688951006 >> 172.21.15.66:0/1688951006 conn(0x7f2e180b1950 msgr2=0x7f2e180b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:32.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/1688951006 shutdown_connections 2024-09-10T22:38:32.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.063+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/1688951006 wait complete. 2024-09-10T22:38:32.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.064+0000 7f2e1f2ef640 1 Processor -- start 2024-09-10T22:38:32.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.064+0000 7f2e1f2ef640 1 -- start start 2024-09-10T22:38:32.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1f2ef640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e18072810 0x7f2e181533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1f2ef640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e180ca4b0 0x7f2e180c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1f2ef640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e18152a40 0x7f2e180c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1f2ef640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180708b0 con 0x7f2e18152a40 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1f2ef640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e18070730 con 0x7f2e180ca4b0 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1f2ef640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2e18070a30 con 0x7f2e18072810 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e18152a40 0x7f2e180c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e18152a40 0x7f2e180c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34584/0 (socket says 172.21.15.66:34584) 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 -- 172.21.15.66:0/2705416221 learned_addr learned my addr 172.21.15.66:0/2705416221 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d08b640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e18072810 0x7f2e181533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1c88a640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e180ca4b0 0x7f2e180c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 -- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e18072810 msgr2=0x7f2e181533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e18072810 0x7f2e181533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 -- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e180ca4b0 msgr2=0x7f2e180c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e180ca4b0 0x7f2e180c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 -- 172.21.15.66:0/2705416221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e18153af0 con 0x7f2e18152a40 2024-09-10T22:38:32.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1c88a640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e180ca4b0 0x7f2e180c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.065+0000 7f2e1d88c640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e18152a40 0x7f2e180c5b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e10006010 tx=0x7f2e10032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.066+0000 7f2e0dffb640 1 -- 172.21.15.66:0/2705416221 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10037040 con 0x7f2e18152a40 2024-09-10T22:38:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.066+0000 7f2e0dffb640 1 -- 172.21.15.66:0/2705416221 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e10042070 con 0x7f2e18152a40 2024-09-10T22:38:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.066+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e18155c10 con 0x7f2e18152a40 2024-09-10T22:38:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.066+0000 7f2e0dffb640 1 -- 172.21.15.66:0/2705416221 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e100042a0 con 0x7f2e18152a40 2024-09-10T22:38:32.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.066+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e181561d0 con 0x7f2e18152a40 2024-09-10T22:38:32.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.067+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e18156410 con 0x7f2e18152a40 2024-09-10T22:38:32.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.068+0000 7f2e0dffb640 1 -- 172.21.15.66:0/2705416221 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2e1003a030 con 0x7f2e18152a40 2024-09-10T22:38:32.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.068+0000 7f2e0dffb640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2dfc077fa0 0x7f2dfc07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.068+0000 7f2e0dffb640 1 -- 172.21.15.66:0/2705416221 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f2e100ba400 con 0x7f2e18152a40 2024-09-10T22:38:32.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.069+0000 7f2e1d08b640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2dfc077fa0 0x7f2dfc07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.069+0000 7f2e1d08b640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2dfc077fa0 0x7f2dfc07a460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400bb50 tx=0x7f2e1400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:32.074 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.073+0000 7f2e0dffb640 1 -- 172.21.15.66:0/2705416221 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2e18156410 con 0x7f2e18152a40 2024-09-10T22:38:32.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.328+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) -- 0x7f2e180ca920 con 0x7f2e18152a40 2024-09-10T22:38:32.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:38:32.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:38:32.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.537+0000 7f2e0dffb640 1 -- 172.21.15.66:0/2705416221 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v153) ==== 147+0+0 (secure 0 0 0) 0x7f2e100831a0 con 0x7f2e18152a40 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2dfc077fa0 msgr2=0x7f2dfc07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2dfc077fa0 0x7f2dfc07a460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400bb50 tx=0x7f2e1400b2e0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e18152a40 msgr2=0x7f2e180c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e18152a40 0x7f2e180c5b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e10006010 tx=0x7f2e10032c90 comp rx=0 tx=0).stop 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 shutdown_connections 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2dfc077fa0 0x7f2dfc07a460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2e18152a40 0x7f2e180c5b60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e180ca4b0 0x7f2e180c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 --2- 172.21.15.66:0/2705416221 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2e18072810 0x7f2e181533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 >> 172.21.15.66:0/2705416221 conn(0x7f2e180b1950 msgr2=0x7f2e180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 shutdown_connections 2024-09-10T22:38:32.542 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.539+0000 7f2e1f2ef640 1 -- 172.21.15.66:0/2705416221 wait complete. 2024-09-10T22:38:32.553 INFO:tasks.cephfs_test_runner:The maximum number of openfiles omap objects keys are now equal to ... ok 2024-09-10T22:38:32.553 INFO:tasks.cephfs_test_runner:Starting test: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) Opening a file should increment omap_total_updates by 1. 2024-09-10T22:38:32.553 INFO:tasks.cephfs_test_runner:test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-09-10T22:38:32.554 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:38:32.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 -- 172.21.15.66:0/394451186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0cdb60 msgr2=0x7ffb2c15f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 --2- 172.21.15.66:0/394451186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0cdb60 0x7ffb2c15f000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0767c0 tx=0x7ffb2402aa70 comp rx=0 tx=0).stop 2024-09-10T22:38:32.659 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 -- 172.21.15.66:0/394451186 shutdown_connections 2024-09-10T22:38:32.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 --2- 172.21.15.66:0/394451186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0cdb60 0x7ffb2c15f000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 --2- 172.21.15.66:0/394451186 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2c0ce830 0x7ffb2c0b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 -- 172.21.15.66:0/394451186 >> 172.21.15.66:0/394451186 conn(0x7ffb2c0d1e30 msgr2=0x7ffb2c0d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:32.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 -- 172.21.15.66:0/394451186 shutdown_connections 2024-09-10T22:38:32.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.657+0000 7ffb32239640 1 -- 172.21.15.66:0/394451186 wait complete. 2024-09-10T22:38:32.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.658+0000 7ffb32239640 1 Processor -- start 2024-09-10T22:38:32.660 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.658+0000 7ffb32239640 1 -- start start 2024-09-10T22:38:32.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb32239640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c0cdb60 0x7ffb2c0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.662 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb32239640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0ce830 0x7ffb2c1f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb32239640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2c06eb40 0x7ffb2c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb32239640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c0b36a0 con 0x7ffb2c0ce830 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb32239640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c0d2ba0 con 0x7ffb2c0cdb60 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb32239640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2c0cec10 con 0x7ffb2c06eb40 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb2affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0ce830 0x7ffb2c1f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb2affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0ce830 0x7ffb2c1f6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34604/0 (socket says 172.21.15.66:34604) 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb2affd640 1 -- 172.21.15.66:0/1574534238 learned_addr learned my addr 172.21.15.66:0/1574534238 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb2bfff640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2c06eb40 0x7ffb2c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.659+0000 7ffb2b7fe640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c0cdb60 0x7ffb2c0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb2affd640 1 -- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2c06eb40 msgr2=0x7ffb2c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb2affd640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2c06eb40 0x7ffb2c0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb2affd640 1 -- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c0cdb60 msgr2=0x7ffb2c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb2affd640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c0cdb60 0x7ffb2c0ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.663 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb2affd640 1 -- 172.21.15.66:0/1574534238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c06efd0 con 0x7ffb2c0ce830 2024-09-10T22:38:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb2affd640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0ce830 0x7ffb2c1f6120 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c400 tx=0x7ffb2000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb13fff640 1 -- 172.21.15.66:0/1574534238 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20011790 con 0x7ffb2c0ce830 2024-09-10T22:38:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb13fff640 1 -- 172.21.15.66:0/1574534238 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb20017070 con 0x7ffb2c0ce830 2024-09-10T22:38:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2c0c9260 con 0x7ffb2c0ce830 2024-09-10T22:38:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb13fff640 1 -- 172.21.15.66:0/1574534238 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20011d00 con 0x7ffb2c0ce830 2024-09-10T22:38:32.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.660+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2c0c9820 con 0x7ffb2c0ce830 2024-09-10T22:38:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.661+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2c0b4c80 con 0x7ffb2c0ce830 2024-09-10T22:38:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.662+0000 7ffb13fff640 1 -- 172.21.15.66:0/1574534238 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffb2001c460 con 0x7ffb2c0ce830 2024-09-10T22:38:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.662+0000 7ffb13fff640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb18077f70 0x7ffb1807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.662+0000 7ffb13fff640 1 -- 172.21.15.66:0/1574534238 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7ffb20099860 con 0x7ffb2c0ce830 2024-09-10T22:38:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.663+0000 7ffb2b7fe640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb18077f70 0x7ffb1807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.663+0000 7ffb2b7fe640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb18077f70 0x7ffb1807a430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c005fd0 tx=0x7ffb1c005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:32.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.667+0000 7ffb13fff640 1 -- 172.21.15.66:0/1574534238 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ffb20062690 con 0x7ffb2c0ce830 2024-09-10T22:38:32.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.859+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb2c0d1160 con 0x7ffb2c0ce830 2024-09-10T22:38:32.861 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.860+0000 7ffb13fff640 1 -- 172.21.15.66:0/1574534238 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13136 (secure 0 0 0) 0x7ffb20061f90 con 0x7ffb2c0ce830 2024-09-10T22:38:32.862 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:32.862 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:38:31.532713+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6834/956430443":"2024-09-11T22:38:28.524345+0000","172.21.15.66:6835/1970715211":"2024-09-11T22:35:52.685163+0000","172.21.15.66:6832/956430443":"2024-09-11T22:38:28.524345+0000","172.21.15.66:6833/1970715211":"2024-09-11T22:35:52.685163+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:38:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.862+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb18077f70 msgr2=0x7ffb1807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb18077f70 0x7ffb1807a430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c005fd0 tx=0x7ffb1c005850 comp rx=0 tx=0).stop 2024-09-10T22:38:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0ce830 msgr2=0x7ffb2c1f6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0ce830 0x7ffb2c1f6120 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c400 tx=0x7ffb2000c8c0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 shutdown_connections 2024-09-10T22:38:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffb18077f70 0x7ffb1807a430 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.865 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffb2c06eb40 0x7ffb2c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb2c0ce830 0x7ffb2c1f6120 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 --2- 172.21.15.66:0/1574534238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c0cdb60 0x7ffb2c0ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 >> 172.21.15.66:0/1574534238 conn(0x7ffb2c0d1e30 msgr2=0x7ffb2c15e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 shutdown_connections 2024-09-10T22:38:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.863+0000 7ffb32239640 1 -- 172.21.15.66:0/1574534238 wait complete. 2024-09-10T22:38:32.877 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T22:34:50.607631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T22:38:32.877 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:38:32.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2717022421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280cac00 msgr2=0x7fc0280cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2717022421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280cac00 0x7fc0280cc840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc02815b1d0 tx=0x7fc01c02a5f0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.985 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2717022421 shutdown_connections 2024-09-10T22:38:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2717022421 >> v2:172.21.15.94:3300/0 conn(0x7fc0280ccd80 0x7fc028073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2717022421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280cac00 0x7fc0280cc840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2717022421 >> 172.21.15.66:0/2717022421 conn(0x7fc0280b1d50 msgr2=0x7fc0280b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2717022421 shutdown_connections 2024-09-10T22:38:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.983+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2717022421 wait complete. 2024-09-10T22:38:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.984+0000 7fc02e4bd640 1 Processor -- start 2024-09-10T22:38:32.986 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.984+0000 7fc02e4bd640 1 -- start start 2024-09-10T22:38:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc02e4bd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0280cac00 0x7fc028146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc02e4bd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280ccd80 0x7fc02815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc02e4bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc028152a10 0x7fc0281571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc02e4bd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc0280740d0 con 0x7fc0280ccd80 2024-09-10T22:38:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc02e4bd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc028073f50 con 0x7fc028152a10 2024-09-10T22:38:32.989 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc02e4bd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc028074250 con 0x7fc0280cac00 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280ccd80 0x7fc02815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280ccd80 0x7fc02815e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34618/0 (socket says 172.21.15.66:34618) 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc0277fe640 1 -- 172.21.15.66:0/2689421985 learned_addr learned my addr 172.21.15.66:0/2689421985 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc02ca5a640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc028152a10 0x7fc0281571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.985+0000 7fc027fff640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0280cac00 0x7fc028146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc0277fe640 1 -- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0280cac00 msgr2=0x7fc028146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc0277fe640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0280cac00 0x7fc028146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc0277fe640 1 -- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc028152a10 msgr2=0x7fc0281571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc0277fe640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc028152a10 0x7fc0281571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc0277fe640 1 -- 172.21.15.66:0/2689421985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02815d6c0 con 0x7fc0280ccd80 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc0277fe640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280ccd80 0x7fc02815e760 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009850 tx=0x7fc01c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc024ff9640 1 -- 172.21.15.66:0/2689421985 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c03f070 con 0x7fc0280ccd80 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc024ff9640 1 -- 172.21.15.66:0/2689421985 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc01c0047a0 con 0x7fc0280ccd80 2024-09-10T22:38:32.990 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02815d9a0 con 0x7fc0280ccd80 2024-09-10T22:38:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc024ff9640 1 -- 172.21.15.66:0/2689421985 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c035030 con 0x7fc0280ccd80 2024-09-10T22:38:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.986+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02815df60 con 0x7fc0280ccd80 2024-09-10T22:38:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.987+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc028074700 con 0x7fc0280ccd80 2024-09-10T22:38:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.988+0000 7fc024ff9640 1 -- 172.21.15.66:0/2689421985 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc01c004bb0 con 0x7fc0280ccd80 2024-09-10T22:38:32.991 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.988+0000 7fc024ff9640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc014077f70 0x7fc01407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:32.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.989+0000 7fc024ff9640 1 -- 172.21.15.66:0/2689421985 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fc01c0bab60 con 0x7fc0280ccd80 2024-09-10T22:38:32.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.989+0000 7fc027fff640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc014077f70 0x7fc01407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:32.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.989+0000 7fc027fff640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc014077f70 0x7fc01407a430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc018001720 tx=0x7fc018009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:32.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:32.993+0000 7fc024ff9640 1 -- 172.21.15.66:0/2689421985 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc01c083340 con 0x7fc0280ccd80 2024-09-10T22:38:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.176+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc028074c40 con 0x7fc0280ccd80 2024-09-10T22:38:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.177+0000 7fc024ff9640 1 -- 172.21.15.66:0/2689421985 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7fc01c087d70 con 0x7fc0280ccd80 2024-09-10T22:38:33.179 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc014077f70 msgr2=0x7fc01407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc014077f70 0x7fc01407a430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc018001720 tx=0x7fc018009040 comp rx=0 tx=0).stop 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280ccd80 msgr2=0x7fc02815e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280ccd80 0x7fc02815e760 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009850 tx=0x7fc01c003910 comp rx=0 tx=0).stop 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 shutdown_connections 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc014077f70 0x7fc01407a430 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc028152a10 0x7fc0281571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc0280ccd80 0x7fc02815e760 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 --2- 172.21.15.66:0/2689421985 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0280cac00 0x7fc028146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 >> 172.21.15.66:0/2689421985 conn(0x7fc0280b1d50 msgr2=0x7fc0280b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 shutdown_connections 2024-09-10T22:38:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.179+0000 7fc02e4bd640 1 -- 172.21.15.66:0/2689421985 wait complete. 2024-09-10T22:38:33.192 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-09-10T22:38:33.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 -- 172.21.15.66:0/2397470811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 msgr2=0x7f38380cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:33.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 --2- 172.21.15.66:0/2397470811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38380cf680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3838161b90 tx=0x7f382802a530 comp rx=0 tx=0).stop 2024-09-10T22:38:33.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 -- 172.21.15.66:0/2397470811 shutdown_connections 2024-09-10T22:38:33.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 --2- 172.21.15.66:0/2397470811 >> v2:172.21.15.94:3300/0 conn(0x7f38380d2d70 0x7f38380d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 --2- 172.21.15.66:0/2397470811 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38380cf680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 -- 172.21.15.66:0/2397470811 >> 172.21.15.66:0/2397470811 conn(0x7f38380d4060 msgr2=0x7f38380d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:33.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 -- 172.21.15.66:0/2397470811 shutdown_connections 2024-09-10T22:38:33.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.295+0000 7f383d471640 1 -- 172.21.15.66:0/2397470811 wait complete. 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 Processor -- start 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 -- start start 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38380cfbc0 0x7f383809ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38381574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38380d2d70 0x7f38380cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f38380d1610 con 0x7f38380d07a0 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38380d1490 con 0x7f38380d2d70 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f383d471640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f38380b40d0 con 0x7f38380cfbc0 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f38367fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38381574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f38367fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38381574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34636/0 (socket says 172.21.15.66:34636) 2024-09-10T22:38:33.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.297+0000 7f38367fc640 1 -- 172.21.15.66:0/713021804 learned_addr learned my addr 172.21.15.66:0/713021804 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f3836ffd640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38380cfbc0 0x7f383809ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f38377fe640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38380d2d70 0x7f38380cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f38367fc640 1 -- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38380cfbc0 msgr2=0x7f383809ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f38367fc640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38380cfbc0 0x7f383809ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f38367fc640 1 -- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38380d2d70 msgr2=0x7f38380cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f38367fc640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38380d2d70 0x7f38380cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f38367fc640 1 -- 172.21.15.66:0/713021804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383809f4c0 con 0x7f38380d07a0 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f38367fc640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38381574a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f382000c970 tx=0x7f382000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f382f7fe640 1 -- 172.21.15.66:0/713021804 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820007bb0 con 0x7f38380d07a0 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f382f7fe640 1 -- 172.21.15.66:0/713021804 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3820007e40 con 0x7f38380d07a0 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38380ccb60 con 0x7f38380d07a0 2024-09-10T22:38:33.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f382f7fe640 1 -- 172.21.15.66:0/713021804 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820005280 con 0x7f38380d07a0 2024-09-10T22:38:33.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.298+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38380cd120 con 0x7f38380d07a0 2024-09-10T22:38:33.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.299+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38380d3e40 con 0x7f38380d07a0 2024-09-10T22:38:33.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.300+0000 7f382f7fe640 1 -- 172.21.15.66:0/713021804 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3820005420 con 0x7f38380d07a0 2024-09-10T22:38:33.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.300+0000 7f382f7fe640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3818077f70 0x7f381807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:33.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.300+0000 7f382f7fe640 1 -- 172.21.15.66:0/713021804 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f3820099df0 con 0x7f38380d07a0 2024-09-10T22:38:33.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.301+0000 7f3836ffd640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3818077f70 0x7f381807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:33.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.301+0000 7f3836ffd640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3818077f70 0x7f381807a430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3828034000 tx=0x7f3828004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:33.306 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.305+0000 7f382f7fe640 1 -- 172.21.15.66:0/713021804 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f383809f4c0 con 0x7f38380d07a0 2024-09-10T22:38:33.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:33.567+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) -- 0x7f38380b7690 con 0x7f38380d07a0 2024-09-10T22:38:34.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.542+0000 7f382f7fe640 1 -- 172.21.15.66:0/713021804 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v155) ==== 146+0+0 (secure 0 0 0) 0x7f3820014030 con 0x7f38380d07a0 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.545+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3818077f70 msgr2=0x7f381807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.545+0000 7f383d471640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3818077f70 0x7f381807a430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3828034000 tx=0x7f3828004730 comp rx=0 tx=0).stop 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.545+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 msgr2=0x7f38381574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.545+0000 7f383d471640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38381574a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f382000c970 tx=0x7f382000ce30 comp rx=0 tx=0).stop 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 shutdown_connections 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f3818077f70 0x7f381807a430 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38380d2d70 0x7f38380cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f38380d07a0 0x7f38381574a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.548 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 --2- 172.21.15.66:0/713021804 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f38380cfbc0 0x7f383809ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 >> 172.21.15.66:0/713021804 conn(0x7f38380d4060 msgr2=0x7f38380b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:34.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 shutdown_connections 2024-09-10T22:38:34.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.546+0000 7f383d471640 1 -- 172.21.15.66:0/713021804 wait complete. 2024-09-10T22:38:34.561 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T22:38:34.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 -- 172.21.15.66:0/876960469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d10c0 msgr2=0x7f5ba4167c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/876960469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d10c0 0x7f5ba4167c60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c005fd0 tx=0x7f5b9c02aa00 comp rx=0 tx=0).stop 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 -- 172.21.15.66:0/876960469 shutdown_connections 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/876960469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d10c0 0x7f5ba4167c60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/876960469 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ba40d2d70 0x7f5ba40d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 -- 172.21.15.66:0/876960469 >> 172.21.15.66:0/876960469 conn(0x7f5ba40d4060 msgr2=0x7f5ba40d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 -- 172.21.15.66:0/876960469 shutdown_connections 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.671+0000 7f5ba99c2640 1 -- 172.21.15.66:0/876960469 wait complete. 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.672+0000 7f5ba99c2640 1 Processor -- start 2024-09-10T22:38:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba99c2640 1 -- start start 2024-09-10T22:38:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba99c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ba40d07a0 0x7f5ba40ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba99c2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ba40d10c0 0x7f5ba4151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba99c2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d2d70 0x7f5ba40c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba99c2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4168320 con 0x7f5ba40d2d70 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba99c2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41681a0 con 0x7f5ba40d07a0 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba99c2640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba41684a0 con 0x7f5ba40d10c0 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d2d70 0x7f5ba40c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d2d70 0x7f5ba40c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37864/0 (socket says 172.21.15.66:37864) 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.673+0000 7f5ba37fe640 1 -- 172.21.15.66:0/3144423912 learned_addr learned my addr 172.21.15.66:0/3144423912 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba2ffd640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ba40d07a0 0x7f5ba40ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5b9a7fc640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ba40d10c0 0x7f5ba4151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba37fe640 1 -- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ba40d10c0 msgr2=0x7f5ba4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba37fe640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ba40d10c0 0x7f5ba4151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba37fe640 1 -- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ba40d07a0 msgr2=0x7f5ba40ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba37fe640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ba40d07a0 0x7f5ba40ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba37fe640 1 -- 172.21.15.66:0/3144423912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba40cd400 con 0x7f5ba40d2d70 2024-09-10T22:38:34.677 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5b9a7fc640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ba40d10c0 0x7f5ba4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:34.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba2ffd640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ba40d07a0 0x7f5ba40ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:34.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba37fe640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d2d70 0x7f5ba40c7b70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c006010 tx=0x7f5b9c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:34.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba0ff9640 1 -- 172.21.15.66:0/3144423912 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c047070 con 0x7f5ba40d2d70 2024-09-10T22:38:34.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba0ff9640 1 -- 172.21.15.66:0/3144423912 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b9c004250 con 0x7f5ba40d2d70 2024-09-10T22:38:34.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba40cd6e0 con 0x7f5ba40d2d70 2024-09-10T22:38:34.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba0ff9640 1 -- 172.21.15.66:0/3144423912 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c033050 con 0x7f5ba40d2d70 2024-09-10T22:38:34.678 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.674+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba4150b50 con 0x7f5ba40d2d70 2024-09-10T22:38:34.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.675+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba4151050 con 0x7f5ba40d2d70 2024-09-10T22:38:34.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.676+0000 7f5ba0ff9640 1 -- 172.21.15.66:0/3144423912 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5b9c004680 con 0x7f5ba40d2d70 2024-09-10T22:38:34.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.676+0000 7f5ba0ff9640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b84077f70 0x7f5b8407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.676+0000 7f5ba0ff9640 1 -- 172.21.15.66:0/3144423912 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f5b9c0bb010 con 0x7f5ba40d2d70 2024-09-10T22:38:34.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.676+0000 7f5ba2ffd640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b84077f70 0x7f5b8407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.679 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.677+0000 7f5ba2ffd640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b84077f70 0x7f5b8407a430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40cbf70 tx=0x7f5b94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:34.682 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.680+0000 7f5ba0ff9640 1 -- 172.21.15.66:0/3144423912 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5ba4151050 con 0x7f5ba40d2d70 2024-09-10T22:38:34.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.867+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f5ba40ca9f0 con 0x7f5ba40d2d70 2024-09-10T22:38:34.869 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.867+0000 7f5ba0ff9640 1 -- 172.21.15.66:0/3144423912 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v29) ==== 59+0+1998 (secure 0 0 0) 0x7f5b9c083db0 con 0x7f5ba40d2d70 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 5584.179480 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4284.998881 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4476.318438 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4656.597806 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5031.347720 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4753.193614 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5217.773028 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4968.712004 2024-09-10T22:38:34.870 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds 20/20 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced debug_ms 2/2 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_frag true 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_debug_scatterstat true 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds dev mds_verify_scatter true 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced client_mount_timeout 600 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_client 20/20 2024-09-10T22:38:34.871 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced debug_ms 1/1 2024-09-10T22:38:34.872 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T22:38:34.872 INFO:teuthology.orchestra.run.smithi066.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b84077f70 msgr2=0x7f5b8407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b84077f70 0x7f5b8407a430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40cbf70 tx=0x7f5b94009040 comp rx=0 tx=0).stop 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d2d70 msgr2=0x7f5ba40c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d2d70 0x7f5ba40c7b70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c006010 tx=0x7f5b9c003f10 comp rx=0 tx=0).stop 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 shutdown_connections 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5b84077f70 0x7f5b8407a430 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ba40d2d70 0x7f5ba40c7b70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5ba40d10c0 0x7f5ba4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 --2- 172.21.15.66:0/3144423912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ba40d07a0 0x7f5ba40ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 >> 172.21.15.66:0/3144423912 conn(0x7f5ba40d4060 msgr2=0x7f5ba40b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:34.873 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 shutdown_connections 2024-09-10T22:38:34.874 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.869+0000 7f5ba99c2640 1 -- 172.21.15.66:0/3144423912 wait complete. 2024-09-10T22:38:34.882 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 -- 172.21.15.66:0/2697252000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40cd4e0 msgr2=0x7f50b40ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/2697252000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40cd4e0 0x7f50b40ce3b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f50b40c7510 tx=0x7f50a802a560 comp rx=0 tx=0).stop 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 -- 172.21.15.66:0/2697252000 shutdown_connections 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/2697252000 >> v2:172.21.15.94:3300/0 conn(0x7f50b40b1440 0x7f50b40cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/2697252000 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40cd4e0 0x7f50b40ce3b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 -- 172.21.15.66:0/2697252000 >> 172.21.15.66:0/2697252000 conn(0x7f50b4074290 msgr2=0x7f50b40746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 -- 172.21.15.66:0/2697252000 shutdown_connections 2024-09-10T22:38:34.992 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.989+0000 7f50ba7f8640 1 -- 172.21.15.66:0/2697252000 wait complete. 2024-09-10T22:38:34.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.991+0000 7f50ba7f8640 1 Processor -- start 2024-09-10T22:38:34.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.991+0000 7f50ba7f8640 1 -- start start 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40b1440 0x7f50b41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b40cd4e0 0x7f50b40c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b415e5d0 0x7f50b40c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50b40cb620 con 0x7f50b40b1440 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f50b40cb4a0 con 0x7f50b40cd4e0 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f50b40cb7a0 con 0x7f50b415e5d0 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40b1440 0x7f50b41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40b1440 0x7f50b41582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37880/0 (socket says 172.21.15.66:37880) 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 -- 172.21.15.66:0/4225315458 learned_addr learned my addr 172.21.15.66:0/4225315458 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b8d95640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b415e5d0 0x7f50b40c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b37fe640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b40cd4e0 0x7f50b40c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.995 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 -- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b415e5d0 msgr2=0x7f50b40c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b415e5d0 0x7f50b40c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 -- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b40cd4e0 msgr2=0x7f50b40c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b40cd4e0 0x7f50b40c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 -- 172.21.15.66:0/4225315458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b40c7720 con 0x7f50b40b1440 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b37fe640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b40cd4e0 0x7f50b40c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b3fff640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40b1440 0x7f50b41582e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b3f0 tx=0x7f50a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b0ff9640 1 -- 172.21.15.66:0/4225315458 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4015020 con 0x7f50b40b1440 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b0ff9640 1 -- 172.21.15.66:0/4225315458 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a4011920 con 0x7f50b40b1440 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b40c79c0 con 0x7f50b40b1440 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50b0ff9640 1 -- 172.21.15.66:0/4225315458 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4011d70 con 0x7f50b40b1440 2024-09-10T22:38:34.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.992+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b40c7f80 con 0x7f50b40b1440 2024-09-10T22:38:34.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.993+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b40cad50 con 0x7f50b40b1440 2024-09-10T22:38:34.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.994+0000 7f50b0ff9640 1 -- 172.21.15.66:0/4225315458 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f50a401a400 con 0x7f50b40b1440 2024-09-10T22:38:34.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.994+0000 7f50b0ff9640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f50a0077f70 0x7f50a007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:34.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.995+0000 7f50b0ff9640 1 -- 172.21.15.66:0/4225315458 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7f50a40999a0 con 0x7f50b40b1440 2024-09-10T22:38:34.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.995+0000 7f50b37fe640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f50a0077f70 0x7f50a007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:34.998 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.995+0000 7f50b37fe640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f50a0077f70 0x7f50a007a430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50a8033000 tx=0x7f50a8004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:35.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:34.999+0000 7f50b0ff9640 1 -- 172.21.15.66:0/4225315458 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f50a401f030 con 0x7f50b40b1440 2024-09-10T22:38:35.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.186+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f50b415c800 con 0x7f50b40b1440 2024-09-10T22:38:35.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.189+0000 7f50b0ff9640 1 -- 172.21.15.66:0/4225315458 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f50a40626c0 con 0x7f50b40b1440 2024-09-10T22:38:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f50a0077f70 msgr2=0x7f50a007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f50a0077f70 0x7f50a007a430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f50a8033000 tx=0x7f50a8004730 comp rx=0 tx=0).stop 2024-09-10T22:38:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40b1440 msgr2=0x7f50b41582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40b1440 0x7f50b41582e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b3f0 tx=0x7f50a400b8b0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 shutdown_connections 2024-09-10T22:38:35.194 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f50a0077f70 0x7f50a007a430 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f50b415e5d0 0x7f50b40c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f50b40cd4e0 0x7f50b40c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 --2- 172.21.15.66:0/4225315458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50b40b1440 0x7f50b41582e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 >> 172.21.15.66:0/4225315458 conn(0x7f50b4074290 msgr2=0x7f50b415cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 shutdown_connections 2024-09-10T22:38:35.195 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.192+0000 7f50ba7f8640 1 -- 172.21.15.66:0/4225315458 wait complete. 2024-09-10T22:38:35.204 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:38:35.205 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:38:35.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:38:35.297 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:38:35.297 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:35.297 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:35.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=930 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:38:35.314 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:38:35.314 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:35.339 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:38:35.379 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:35.379 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:35.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=930 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:38:35.395 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:38:35.395 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:35.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:38:35.414 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T22:38:35.414 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 -- 172.21.15.66:0/1139919994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700b51c0 msgr2=0x7fae700cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 --2- 172.21.15.66:0/1139919994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700b51c0 0x7fae700cd450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fae70073f10 tx=0x7fae6402a5c0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 -- 172.21.15.66:0/1139919994 shutdown_connections 2024-09-10T22:38:35.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 --2- 172.21.15.66:0/1139919994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700b51c0 0x7fae700cd450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 --2- 172.21.15.66:0/1139919994 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae700ce830 0x7fae700b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 -- 172.21.15.66:0/1139919994 >> 172.21.15.66:0/1139919994 conn(0x7fae700d1e30 msgr2=0x7fae700d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:35.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 -- 172.21.15.66:0/1139919994 shutdown_connections 2024-09-10T22:38:35.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.538+0000 7fae75e52640 1 -- 172.21.15.66:0/1139919994 wait complete. 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.539+0000 7fae75e52640 1 Processor -- start 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.539+0000 7fae75e52640 1 -- start start 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae75e52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae700ce830 0x7fae700cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae75e52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700d2950 0x7fae70155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae75e52640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae701f6f80 0x7fae700b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae75e52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fae700cedd0 con 0x7fae700d2950 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae75e52640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae700cec50 con 0x7fae700ce830 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae75e52640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fae700cef50 con 0x7fae701f6f80 2024-09-10T22:38:35.543 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700d2950 0x7fae70155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700d2950 0x7fae70155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37886/0 (socket says 172.21.15.66:37886) 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 -- 172.21.15.66:0/19006336 learned_addr learned my addr 172.21.15.66:0/19006336 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6f7fe640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae700ce830 0x7fae700cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6ffff640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae701f6f80 0x7fae700b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 -- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae701f6f80 msgr2=0x7fae700b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae701f6f80 0x7fae700b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 -- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae700ce830 msgr2=0x7fae700cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae700ce830 0x7fae700cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.540+0000 7fae6effd640 1 -- 172.21.15.66:0/19006336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae70155680 con 0x7fae700d2950 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.541+0000 7fae6effd640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700d2950 0x7fae70155140 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fae640099c0 tx=0x7fae64003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.541+0000 7fae57fff640 1 -- 172.21.15.66:0/19006336 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6403f070 con 0x7fae700d2950 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.541+0000 7fae57fff640 1 -- 172.21.15.66:0/19006336 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae640047a0 con 0x7fae700d2950 2024-09-10T22:38:35.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.541+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae700cae90 con 0x7fae700d2950 2024-09-10T22:38:35.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.541+0000 7fae57fff640 1 -- 172.21.15.66:0/19006336 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64035030 con 0x7fae700d2950 2024-09-10T22:38:35.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.541+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae700ca3b0 con 0x7fae700d2950 2024-09-10T22:38:35.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.542+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae700b4c80 con 0x7fae700d2950 2024-09-10T22:38:35.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.543+0000 7fae57fff640 1 -- 172.21.15.66:0/19006336 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fae64004bd0 con 0x7fae700d2950 2024-09-10T22:38:35.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.543+0000 7fae57fff640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fae5c077fa0 0x7fae5c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.543+0000 7fae57fff640 1 -- 172.21.15.66:0/19006336 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fae640baf30 con 0x7fae700d2950 2024-09-10T22:38:35.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.543+0000 7fae6f7fe640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fae5c077fa0 0x7fae5c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.546 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.544+0000 7fae6f7fe640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fae5c077fa0 0x7fae5c07a460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae60005fd0 tx=0x7fae6000b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:35.549 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.548+0000 7fae57fff640 1 -- 172.21.15.66:0/19006336 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fae64083680 con 0x7fae700d2950 2024-09-10T22:38:35.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.769+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae700d0620 con 0x7fae700d2950 2024-09-10T22:38:35.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.769+0000 7fae57fff640 1 -- 172.21.15.66:0/19006336 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3905 (secure 0 0 0) 0x7fae640880b0 con 0x7fae700d2950 2024-09-10T22:38:35.771 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:35.772 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":26,"btime":"2024-09-10T22:38:29:526441+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4252749143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4252749143},{"type":"v1","addr":"172.21.15.94:6835","nonce":4252749143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5011,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/58860958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":58860958},{"type":"v1","addr":"172.21.15.94:6837","nonce":58860958}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/3000446890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3000446890},{"type":"v1","addr":"172.21.15.66:6835","nonce":3000446890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/3616493558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":3616493558},{"type":"v1","addr":"172.21.15.66:6834","nonce":3616493558}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-09-10T22:38:35.772 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 26 2024-09-10T22:38:35.773 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fae5c077fa0 msgr2=0x7fae5c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fae5c077fa0 0x7fae5c07a460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae60005fd0 tx=0x7fae6000b390 comp rx=0 tx=0).stop 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700d2950 msgr2=0x7fae70155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700d2950 0x7fae70155140 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fae640099c0 tx=0x7fae64003910 comp rx=0 tx=0).stop 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 shutdown_connections 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fae5c077fa0 0x7fae5c07a460 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fae701f6f80 0x7fae700b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae700d2950 0x7fae70155140 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 --2- 172.21.15.66:0/19006336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae700ce830 0x7fae700cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 >> 172.21.15.66:0/19006336 conn(0x7fae700d1e30 msgr2=0x7fae70157560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 shutdown_connections 2024-09-10T22:38:35.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.771+0000 7fae75e52640 1 -- 172.21.15.66:0/19006336 wait complete. 2024-09-10T22:38:35.784 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:38:35.784 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:38:35.785 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:38:35.785 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:35.785 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:38:35.785 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:38:35.785 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:38:35.785 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:35.786 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:38:35.786 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:38:35.786 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:38:35.786 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:35.786 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:38:35.786 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:38:35.787 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:38:35.787 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:35.794 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:38:35.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:38:35.795 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:38:35.796 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:38:35.799 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:38:35.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:38:35.800 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:38:35.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:38:35.838 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:38:35.838 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:35.839 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:38:35.840 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:35.842 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:38:35.842 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:35.843 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:38:35.843 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:35.843 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T22:38:35.916 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:38:35.916 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:38:35.926 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:38:35.927 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:38:35.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.953+0000 7fcf276e6640 1 -- 172.21.15.66:0/2118448697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 msgr2=0x7fcf200cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.953+0000 7fcf276e6640 1 --2- 172.21.15.66:0/2118448697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200cec20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcf18005fd0 tx=0x7fcf1802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.953+0000 7fcf276e6640 1 -- 172.21.15.66:0/2118448697 shutdown_connections 2024-09-10T22:38:35.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.953+0000 7fcf276e6640 1 --2- 172.21.15.66:0/2118448697 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200cec20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.953+0000 7fcf276e6640 1 -- 172.21.15.66:0/2118448697 >> 172.21.15.66:0/2118448697 conn(0x7fcf200d1e30 msgr2=0x7fcf200d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:35.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.953+0000 7fcf276e6640 1 -- 172.21.15.66:0/2118448697 shutdown_connections 2024-09-10T22:38:35.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.953+0000 7fcf276e6640 1 -- 172.21.15.66:0/2118448697 wait complete. 2024-09-10T22:38:35.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.954+0000 7fcf276e6640 1 Processor -- start 2024-09-10T22:38:35.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.954+0000 7fcf276e6640 1 -- start start 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf276e6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf276e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf200ce840 0x7fcf200af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf276e6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf200d2950 0x7fcf200c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf276e6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcf200cf2e0 con 0x7fcf200cd980 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf276e6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf200cf160 con 0x7fcf200ce840 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf276e6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcf200cf460 con 0x7fcf200d2950 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37938/0 (socket says 172.21.15.66:37938) 2024-09-10T22:38:35.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 -- 172.21.15.66:0/1442805820 learned_addr learned my addr 172.21.15.66:0/1442805820 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf24c81640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf200ce840 0x7fcf200af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25c83640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf200d2950 0x7fcf200c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 -- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf200d2950 msgr2=0x7fcf200c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf200d2950 0x7fcf200c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 -- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf200ce840 msgr2=0x7fcf200af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf200ce840 0x7fcf200af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 -- 172.21.15.66:0/1442805820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf200caae0 con 0x7fcf200cd980 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.955+0000 7fcf25482640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200ca5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c500 tx=0x7fcf1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.956+0000 7fcf15ffb640 1 -- 172.21.15.66:0/1442805820 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0052d0 con 0x7fcf200cd980 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.956+0000 7fcf15ffb640 1 -- 172.21.15.66:0/1442805820 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf1c005470 con 0x7fcf200cd980 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.956+0000 7fcf15ffb640 1 -- 172.21.15.66:0/1442805820 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c012870 con 0x7fcf200cd980 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.956+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf201f64e0 con 0x7fcf200cd980 2024-09-10T22:38:35.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.956+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf201f6aa0 con 0x7fcf200cd980 2024-09-10T22:38:35.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.957+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf200cec20 con 0x7fcf200cd980 2024-09-10T22:38:35.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.958+0000 7fcf15ffb640 1 -- 172.21.15.66:0/1442805820 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcf1c0035f0 con 0x7fcf200cd980 2024-09-10T22:38:35.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.958+0000 7fcf15ffb640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcf04077fa0 0x7fcf0407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:35.960 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.958+0000 7fcf15ffb640 1 -- 172.21.15.66:0/1442805820 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5029+0+0 (secure 0 0 0) 0x7fcf1c0994d0 con 0x7fcf200cd980 2024-09-10T22:38:35.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.959+0000 7fcf24c81640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcf04077fa0 0x7fcf0407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:35.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.959+0000 7fcf24c81640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcf04077fa0 0x7fcf0407a460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf100046b0 tx=0x7fcf10007cc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:35.964 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:35.963+0000 7fcf15ffb640 1 -- 172.21.15.66:0/1442805820 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcf1c061c30 con 0x7fcf200cd980 2024-09-10T22:38:36.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.160+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcf200b7db0 con 0x7fcf200cd980 2024-09-10T22:38:36.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.564+0000 7fcf15ffb640 1 -- 172.21.15.66:0/1442805820 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7fcf1c066660 con 0x7fcf200cd980 2024-09-10T22:38:36.566 INFO:teuthology.orchestra.run.smithi066.stderr: removed all blocklist entries 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcf04077fa0 msgr2=0x7fcf0407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcf04077fa0 0x7fcf0407a460 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf100046b0 tx=0x7fcf10007cc0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 msgr2=0x7fcf200ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200ca5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c500 tx=0x7fcf1c00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 shutdown_connections 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcf04077fa0 0x7fcf0407a460 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcf200d2950 0x7fcf200c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf200ce840 0x7fcf200af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 --2- 172.21.15.66:0/1442805820 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcf200cd980 0x7fcf200ca5a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 >> 172.21.15.66:0/1442805820 conn(0x7fcf200d1e30 msgr2=0x7fcf200b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.566+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 shutdown_connections 2024-09-10T22:38:36.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.567+0000 7fcf276e6640 1 -- 172.21.15.66:0/1442805820 wait complete. 2024-09-10T22:38:36.584 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T22:38:36.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.689+0000 7fe94a008640 1 -- 172.21.15.66:0/1122545660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c004d40 msgr2=0x7fe92c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:36.691 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.689+0000 7fe94a008640 1 --2- 172.21.15.66:0/1122545660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe92c004d40 0x7fe92c0051c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009ed0 tx=0x7fe92c005830 comp rx=0 tx=0).stop 2024-09-10T22:38:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.689+0000 7fe943fff640 1 -- 172.21.15.66:0/1122545660 reap_dead start 2024-09-10T22:38:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.689+0000 7fe94a008640 1 -- 172.21.15.66:0/1122545660 shutdown_connections 2024-09-10T22:38:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.689+0000 7fe94a008640 1 -- 172.21.15.66:0/1122545660 >> 172.21.15.66:0/1122545660 conn(0x7fe9440d4060 msgr2=0x7fe9440d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.689+0000 7fe94a008640 1 -- 172.21.15.66:0/1122545660 shutdown_connections 2024-09-10T22:38:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.689+0000 7fe94a008640 1 -- 172.21.15.66:0/1122545660 wait complete. 2024-09-10T22:38:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.690+0000 7fe94a008640 1 Processor -- start 2024-09-10T22:38:36.692 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.690+0000 7fe94a008640 1 -- start start 2024-09-10T22:38:36.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe94a008640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe92c004d40 0x7fe9440b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:36.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe94a008640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9440b2300 0x7fe9440b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:36.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe94a008640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441f5240 0x7fe9441f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:36.694 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe94a008640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440d0d00 con 0x7fe9440b2300 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe94a008640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440d0b80 con 0x7fe9441f5240 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe94a008640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe9440d0e80 con 0x7fe92c004d40 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe942ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9440b2300 0x7fe9440b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe942ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9440b2300 0x7fe9440b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37960/0 (socket says 172.21.15.66:37960) 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe942ffd640 1 -- 172.21.15.66:0/649588738 learned_addr learned my addr 172.21.15.66:0/649588738 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe9437fe640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe92c004d40 0x7fe9440b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.691+0000 7fe943fff640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441f5240 0x7fe9441f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe942ffd640 1 -- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe92c004d40 msgr2=0x7fe9440b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe942ffd640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe92c004d40 0x7fe9440b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe942ffd640 1 -- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441f5240 msgr2=0x7fe9441f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe942ffd640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441f5240 0x7fe9441f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe942ffd640 1 -- 172.21.15.66:0/649588738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9440b2cc0 con 0x7fe9440b2300 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe943fff640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441f5240 0x7fe9441f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:36.695 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe942ffd640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9440b2300 0x7fe9440b2780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe938011ab0 tx=0x7fe938011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:36.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe92bfff640 1 -- 172.21.15.66:0/649588738 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93801e070 con 0x7fe9440b2300 2024-09-10T22:38:36.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe92bfff640 1 -- 172.21.15.66:0/649588738 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe93800fbb0 con 0x7fe9440b2300 2024-09-10T22:38:36.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9440b6f30 con 0x7fe9440b2300 2024-09-10T22:38:36.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe92bfff640 1 -- 172.21.15.66:0/649588738 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938019410 con 0x7fe9440b2300 2024-09-10T22:38:36.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.692+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9440b74f0 con 0x7fe9440b2300 2024-09-10T22:38:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.693+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9440d4c10 con 0x7fe9440b2300 2024-09-10T22:38:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.694+0000 7fe92bfff640 1 -- 172.21.15.66:0/649588738 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe93802b080 con 0x7fe9440b2300 2024-09-10T22:38:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.694+0000 7fe92bfff640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe930077f70 0x7fe93007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.694+0000 7fe92bfff640 1 -- 172.21.15.66:0/649588738 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fe93809a8f0 con 0x7fe9440b2300 2024-09-10T22:38:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.695+0000 7fe9437fe640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe930077f70 0x7fe93007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:36.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.695+0000 7fe9437fe640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe930077f70 0x7fe93007a430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe934006cd0 tx=0x7fe9340067a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:36.700 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.698+0000 7fe92bfff640 1 -- 172.21.15.66:0/649588738 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe9440d4c10 con 0x7fe9440b2300 2024-09-10T22:38:36.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.955+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe9440d3e40 con 0x7fe9440b2300 2024-09-10T22:38:36.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.956+0000 7fe92bfff640 1 -- 172.21.15.66:0/649588738 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fe9380631d0 con 0x7fe9440b2300 2024-09-10T22:38:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:36.958 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:38:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "auth_dump": [ 2024-09-10T22:38:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.958 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.a", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmOynNABAAKH0qcVEjgabxdgbcMzFenw==", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.b", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBm92zKDBAAU/37cpMQbv3hwxpOOF7wqw==", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:38:36.959 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.c", 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmJyYSCBAAHOWAMRZJfptyuhaBDmMdHg==", 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.960 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mds.d", 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBm9InwExAAOmJR0PSYaLknUYrjqdx8aQ==", 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.0", 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBfyeBmCe8JNBAANHCDeqvquGRJDSdZEF/nKg==", 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.961 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.1", 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBiyeBmAHFpNhAAZ/NEsvKX+rtDoru1DhdjMQ==", 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.962 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.2", 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBmyeBmVeTpGhAAtIciTFU74AgMICAgqvD0/g==", 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.3", 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBpyeBm/01eLBAAeiodosiLdSg26EyvnT+OeA==", 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.963 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.4", 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBuyeBmyrwOABAAx2o/hEASH4yqt4ZmQA68Wg==", 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.5", 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBxyeBmIZEGAhAAd8+PqJWUZGZyB7BHmz60xw==", 2024-09-10T22:38:36.964 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.6", 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB0yeBmvBX4IBAAH7DccJtEh0K9AYIrb/1GjA==", 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.965 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "osd.7", 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQB4yeBmGD4nBRAAWQ51L9Bx7x6P0dwTRRFe8Q==", 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow profile osd", 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile osd", 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.0", 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmyY04GBAAcxW03/29zpbpxXzE+JrsFA==", 2024-09-10T22:38:36.966 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow r", 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.1", 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmtjE1IRAAO/ItVtJZ2xGcQj/LcTecMQ==", 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.967 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.2", 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmasn2HRAAv24FRmz9CPJsGJ8DA1hOIQ==", 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.3", 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBcyeBmz6LSJhAASEiDnC8gX9owU8E46MLj/w==", 2024-09-10T22:38:36.968 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow r", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow rw", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow rwx" 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "client.admin", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbyeBm8C1oHxAAPrYLLz62iPAQFG0SQ04DjA==", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr": "allow *", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow *", 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.969 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.x", 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbyeBm80AWORAA002Ap5bZLKASMXqSSM15dw==", 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "entity": "mgr.y", 2024-09-10T22:38:36.970 INFO:teuthology.orchestra.run.smithi066.stdout: "key": "AQBbyeBmchE7NRAAGJXMPuxRS+9kbKAHx1KX4A==", 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "caps": { 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "mds": "allow *", 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "mon": "allow profile mgr", 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: "osd": "allow *" 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T22:38:36.971 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:38:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe930077f70 msgr2=0x7fe93007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe930077f70 0x7fe93007a430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe934006cd0 tx=0x7fe9340067a0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9440b2300 msgr2=0x7fe9440b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9440b2300 0x7fe9440b2780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe938011ab0 tx=0x7fe938011eb0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 shutdown_connections 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe930077f70 0x7fe93007a430 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441f5240 0x7fe9441f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9440b2300 0x7fe9440b2780 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 --2- 172.21.15.66:0/649588738 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe92c004d40 0x7fe9440b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.958+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 >> 172.21.15.66:0/649588738 conn(0x7fe9440d4060 msgr2=0x7fe944162910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.959+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 shutdown_connections 2024-09-10T22:38:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:36.959+0000 7fe94a008640 1 -- 172.21.15.66:0/649588738 wait complete. 2024-09-10T22:38:36.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 -- 172.21.15.66:0/385855838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40d07a0 msgr2=0x7f0ec40cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 --2- 172.21.15.66:0/385855838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40d07a0 0x7f0ec40cf680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4161b90 tx=0x7f0eb402a560 comp rx=0 tx=0).stop 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 -- 172.21.15.66:0/385855838 shutdown_connections 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 --2- 172.21.15.66:0/385855838 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ec40cfbc0 0x7f0ec40d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 --2- 172.21.15.66:0/385855838 >> v2:172.21.15.94:3300/0 conn(0x7f0ec40d2d70 0x7f0ec40d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 --2- 172.21.15.66:0/385855838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40d07a0 0x7f0ec40cf680 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 -- 172.21.15.66:0/385855838 >> 172.21.15.66:0/385855838 conn(0x7f0ec40d4060 msgr2=0x7f0ec40d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 -- 172.21.15.66:0/385855838 shutdown_connections 2024-09-10T22:38:37.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.077+0000 7f0eca86a640 1 -- 172.21.15.66:0/385855838 wait complete. 2024-09-10T22:38:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.079+0000 7f0eca86a640 1 Processor -- start 2024-09-10T22:38:37.081 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.079+0000 7f0eca86a640 1 -- start start 2024-09-10T22:38:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40cfbc0 0x7f0ec41576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ec40d07a0 0x7f0ec40c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ec40d2d70 0x7f0ec4159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec40b3d50 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec40b3bd0 con 0x7f0ec40d2d70 2024-09-10T22:38:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec40b3ed0 con 0x7f0ec40d07a0 2024-09-10T22:38:37.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40cfbc0 0x7f0ec41576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40cfbc0 0x7f0ec41576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37966/0 (socket says 172.21.15.66:37966) 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 -- 172.21.15.66:0/2407859264 learned_addr learned my addr 172.21.15.66:0/2407859264 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec37fe640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ec40d07a0 0x7f0ec40c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec8e07640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ec40d2d70 0x7f0ec4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 -- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ec40d07a0 msgr2=0x7f0ec40c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ec40d07a0 0x7f0ec40c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 -- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ec40d2d70 msgr2=0x7f0ec4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ec40d2d70 0x7f0ec4159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 -- 172.21.15.66:0/2407859264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec40ca870 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec8e07640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ec40d2d70 0x7f0ec4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec3fff640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40cfbc0 0x7f0ec41576c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40099c0 tx=0x7f0eb40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:37.084 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec0ff9640 1 -- 172.21.15.66:0/2407859264 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb403f070 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec0ff9640 1 -- 172.21.15.66:0/2407859264 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb4004890 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec4158e20 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0ec0ff9640 1 -- 172.21.15.66:0/2407859264 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4035030 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.085 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.080+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec41593e0 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.081+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec40cf680 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.082+0000 7f0ec0ff9640 1 -- 172.21.15.66:0/2407859264 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0eb4004ce0 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.083+0000 7f0ec0ff9640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0eb0077f70 0x7f0eb007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.083+0000 7f0ec0ff9640 1 -- 172.21.15.66:0/2407859264 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f0eb40baa80 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.083+0000 7f0ec37fe640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0eb0077f70 0x7f0eb007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.084+0000 7f0ec37fe640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0eb0077f70 0x7f0eb007a430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ec415d660 tx=0x7f0eb800a250 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:37.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.087+0000 7f0ec0ff9640 1 -- 172.21.15.66:0/2407859264 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0eb4083310 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.274+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0ec40d3e40 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.275+0000 7f0ec0ff9640 1 -- 172.21.15.66:0/2407859264 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7f0eb4087d40 con 0x7f0ec40cfbc0 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2024-09-10T22:38:37.277 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T22:38:37.278 INFO:teuthology.orchestra.run.smithi066.stdout:] 2024-09-10T22:38:37.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0eb0077f70 msgr2=0x7f0eb007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0eb0077f70 0x7f0eb007a430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0ec415d660 tx=0x7f0eb800a250 comp rx=0 tx=0).stop 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40cfbc0 msgr2=0x7f0ec41576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40cfbc0 0x7f0ec41576c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40099c0 tx=0x7f0eb40039e0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 shutdown_connections 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0eb0077f70 0x7f0eb007a430 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ec40d2d70 0x7f0ec4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ec40d07a0 0x7f0ec40c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 --2- 172.21.15.66:0/2407859264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec40cfbc0 0x7f0ec41576c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 >> 172.21.15.66:0/2407859264 conn(0x7f0ec40d4060 msgr2=0x7f0ec4157f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 shutdown_connections 2024-09-10T22:38:37.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.277+0000 7f0eca86a640 1 -- 172.21.15.66:0/2407859264 wait complete. 2024-09-10T22:38:37.291 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:38:37.292 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:38:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 -- 172.21.15.66:0/2242977179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980d0b00 msgr2=0x7f5c980afd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/2242977179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980d0b00 0x7f5c980afd30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5c90005fd0 tx=0x7f5c9002aa00 comp rx=0 tx=0).stop 2024-09-10T22:38:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 -- 172.21.15.66:0/2242977179 shutdown_connections 2024-09-10T22:38:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/2242977179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c980d0b00 0x7f5c980afd30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/2242977179 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c9815f950 0x7f5c980af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 -- 172.21.15.66:0/2242977179 >> 172.21.15.66:0/2242977179 conn(0x7f5c980d1ed0 msgr2=0x7f5c980d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 -- 172.21.15.66:0/2242977179 shutdown_connections 2024-09-10T22:38:37.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.399+0000 7f5ca02a9640 1 -- 172.21.15.66:0/2242977179 wait complete. 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.400+0000 7f5ca02a9640 1 Processor -- start 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.400+0000 7f5ca02a9640 1 -- start start 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5ca02a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c980d0140 0x7f5c9814f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5ca02a9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980d0b00 0x7f5c980cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5ca02a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9815f950 0x7f5c981f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5ca02a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c980b0c00 con 0x7f5c9815f950 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5ca02a9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c980d0840 con 0x7f5c980d0140 2024-09-10T22:38:37.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5ca02a9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5c980b0d80 con 0x7f5c980d0b00 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9815f950 0x7f5c981f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9815f950 0x7f5c981f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37974/0 (socket says 172.21.15.66:37974) 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 -- 172.21.15.66:0/3179928900 learned_addr learned my addr 172.21.15.66:0/3179928900 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9d844640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980d0b00 0x7f5c980cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e045640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c980d0140 0x7f5c9814f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 -- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980d0b00 msgr2=0x7f5c980cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980d0b00 0x7f5c980cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 -- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c980d0140 msgr2=0x7f5c9814f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c980d0140 0x7f5c9814f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 -- 172.21.15.66:0/3179928900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c981f3860 con 0x7f5c9815f950 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.401+0000 7f5c9e846640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9815f950 0x7f5c981f6260 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009d90 tx=0x7f5c90003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:37.405 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.402+0000 7f5c8effd640 1 -- 172.21.15.66:0/3179928900 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90047070 con 0x7f5c9815f950 2024-09-10T22:38:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.402+0000 7f5c8effd640 1 -- 172.21.15.66:0/3179928900 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c90004250 con 0x7f5c9815f950 2024-09-10T22:38:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.402+0000 7f5c8effd640 1 -- 172.21.15.66:0/3179928900 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90033050 con 0x7f5c9815f950 2024-09-10T22:38:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.402+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c980c65a0 con 0x7f5c9815f950 2024-09-10T22:38:37.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.402+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c980c6b60 con 0x7f5c9815f950 2024-09-10T22:38:37.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.403+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9815f240 con 0x7f5c9815f950 2024-09-10T22:38:37.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.403+0000 7f5c8effd640 1 -- 172.21.15.66:0/3179928900 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5c90004680 con 0x7f5c9815f950 2024-09-10T22:38:37.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.404+0000 7f5c8effd640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c7c077f70 0x7f5c7c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:37.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.404+0000 7f5c8effd640 1 -- 172.21.15.66:0/3179928900 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f5c900baf50 con 0x7f5c9815f950 2024-09-10T22:38:37.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.404+0000 7f5c9e045640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c7c077f70 0x7f5c7c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:37.407 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.405+0000 7f5c9e045640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c7c077f70 0x7f5c7c07a430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c94004ea0 tx=0x7f5c94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:37.410 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.408+0000 7f5c8effd640 1 -- 172.21.15.66:0/3179928900 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5c90083e30 con 0x7f5c9815f950 2024-09-10T22:38:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:37.596+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5c9809fec0 con 0x7f5c9815f950 2024-09-10T22:38:38.580 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.578+0000 7f5c8effd640 1 -- 172.21.15.66:0/3179928900 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7f5c90083730 con 0x7f5c9815f950 2024-09-10T22:38:38.580 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2024-09-10T22:38:38.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c7c077f70 msgr2=0x7f5c7c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:38.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c7c077f70 0x7f5c7c07a430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c94004ea0 tx=0x7f5c94009040 comp rx=0 tx=0).stop 2024-09-10T22:38:38.584 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9815f950 msgr2=0x7f5c981f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9815f950 0x7f5c981f6260 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009d90 tx=0x7f5c90003f10 comp rx=0 tx=0).stop 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 shutdown_connections 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c7c077f70 0x7f5c7c07a430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c9815f950 0x7f5c981f6260 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c980d0b00 0x7f5c980cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 --2- 172.21.15.66:0/3179928900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c980d0140 0x7f5c9814f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 >> 172.21.15.66:0/3179928900 conn(0x7f5c980d1ed0 msgr2=0x7f5c98072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 shutdown_connections 2024-09-10T22:38:38.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.582+0000 7f5ca02a9640 1 -- 172.21.15.66:0/3179928900 wait complete. 2024-09-10T22:38:38.601 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.718+0000 7fe58979b640 1 -- 172.21.15.66:0/1051931125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 msgr2=0x7fe584167c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.718+0000 7fe58979b640 1 --2- 172.21.15.66:0/1051931125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584167c60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009830 tx=0x7fe57c02a510 comp rx=0 tx=0).stop 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.718+0000 7fe58979b640 1 -- 172.21.15.66:0/1051931125 shutdown_connections 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.718+0000 7fe58979b640 1 --2- 172.21.15.66:0/1051931125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584167c60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.718+0000 7fe58979b640 1 --2- 172.21.15.66:0/1051931125 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5840d2d70 0x7fe5840d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.718+0000 7fe58979b640 1 -- 172.21.15.66:0/1051931125 >> 172.21.15.66:0/1051931125 conn(0x7fe5840d4060 msgr2=0x7fe5840d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.719+0000 7fe58979b640 1 -- 172.21.15.66:0/1051931125 shutdown_connections 2024-09-10T22:38:38.721 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.719+0000 7fe58979b640 1 -- 172.21.15.66:0/1051931125 wait complete. 2024-09-10T22:38:38.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.720+0000 7fe58979b640 1 Processor -- start 2024-09-10T22:38:38.722 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.720+0000 7fe58979b640 1 -- start start 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5840d07a0 0x7fe584161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5840d2d70 0x7fe58415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe584168320 con 0x7fe5840d10c0 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5841681a0 con 0x7fe5840d07a0 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe5841684a0 con 0x7fe5840d2d70 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37986/0 (socket says 172.21.15.66:37986) 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 -- 172.21.15.66:0/164401806 learned_addr learned my addr 172.21.15.66:0/164401806 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe582ffd640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5840d07a0 0x7fe584161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:38.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5837fe640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5840d2d70 0x7fe58415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 -- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5840d2d70 msgr2=0x7fe58415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5840d2d70 0x7fe58415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 -- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5840d07a0 msgr2=0x7fe584161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5840d07a0 0x7fe584161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 -- 172.21.15.66:0/164401806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe584161980 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe5827fc640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584156270 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009fd0 tx=0x7fe56c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe57b7fe640 1 -- 172.21.15.66:0/164401806 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00cc60 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe57b7fe640 1 -- 172.21.15.66:0/164401806 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe56c00ce00 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe584157560 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe57b7fe640 1 -- 172.21.15.66:0/164401806 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004280 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.721+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe584157280 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.722+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5840d0b80 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.723+0000 7fe57b7fe640 1 -- 172.21.15.66:0/164401806 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe56c0035f0 con 0x7fe5840d10c0 2024-09-10T22:38:38.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.724+0000 7fe57b7fe640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe564077f70 0x7fe56407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.724+0000 7fe57b7fe640 1 -- 172.21.15.66:0/164401806 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5227+0+0 (secure 0 0 0) 0x7fe56c09a4b0 con 0x7fe5840d10c0 2024-09-10T22:38:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.724+0000 7fe582ffd640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe564077f70 0x7fe56407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:38.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.724+0000 7fe582ffd640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe564077f70 0x7fe56407a430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5840c81e0 tx=0x7fe574009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:38.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.728+0000 7fe57b7fe640 1 -- 172.21.15.66:0/164401806 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe56c063190 con 0x7fe5840d10c0 2024-09-10T22:38:38.921 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:38.919+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe5840d3e40 con 0x7fe5840d10c0 2024-09-10T22:38:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.583+0000 7fe57b7fe640 1 -- 172.21.15.66:0/164401806 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7fe56c062a90 con 0x7fe5840d10c0 2024-09-10T22:38:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe564077f70 msgr2=0x7fe56407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe564077f70 0x7fe56407a430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe5840c81e0 tx=0x7fe574009040 comp rx=0 tx=0).stop 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 msgr2=0x7fe584156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584156270 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009fd0 tx=0x7fe56c00ec20 comp rx=0 tx=0).stop 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 shutdown_connections 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe564077f70 0x7fe56407a430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe5840d2d70 0x7fe58415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5840d10c0 0x7fe584156270 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 --2- 172.21.15.66:0/164401806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5840d07a0 0x7fe584161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.586+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 >> 172.21.15.66:0/164401806 conn(0x7fe5840d4060 msgr2=0x7fe5840b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.587+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 shutdown_connections 2024-09-10T22:38:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.587+0000 7fe58979b640 1 -- 172.21.15.66:0/164401806 wait complete. 2024-09-10T22:38:39.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:38:39.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 -- 172.21.15.66:0/156449020 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 msgr2=0x7fd1400cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:39.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 --2- 172.21.15.66:0/156449020 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 0x7fd1400cf680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd130005fd0 tx=0x7fd13002a5c0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 -- 172.21.15.66:0/156449020 shutdown_connections 2024-09-10T22:38:39.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 --2- 172.21.15.66:0/156449020 >> v2:172.21.15.94:3300/0 conn(0x7fd1400cfbc0 0x7fd140167c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.727 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 --2- 172.21.15.66:0/156449020 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 0x7fd1400cf680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 -- 172.21.15.66:0/156449020 >> 172.21.15.66:0/156449020 conn(0x7fd1400d4060 msgr2=0x7fd1400d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:39.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 -- 172.21.15.66:0/156449020 shutdown_connections 2024-09-10T22:38:39.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.725+0000 7fd144c9b640 1 -- 172.21.15.66:0/156449020 wait complete. 2024-09-10T22:38:39.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.726+0000 7fd144c9b640 1 Processor -- start 2024-09-10T22:38:39.728 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.726+0000 7fd144c9b640 1 -- start start 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd144c9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1400cfbc0 0x7fd14015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd144c9b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 0x7fd1400c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd144c9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1400d2d70 0x7fd1400b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd144c9b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd140168320 con 0x7fd1400cfbc0 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd144c9b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401681a0 con 0x7fd1400d2d70 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd144c9b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd1401684a0 con 0x7fd1400d07a0 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1400cfbc0 0x7fd14015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1400d2d70 0x7fd1400b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13e59c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1400cfbc0 0x7fd14015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37988/0 (socket says 172.21.15.66:37988) 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd137fff640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 0x7fd1400c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13e59c640 1 -- 172.21.15.66:0/1111118837 learned_addr learned my addr 172.21.15.66:0/1111118837 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13ed9d640 1 -- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 msgr2=0x7fd1400c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13ed9d640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 0x7fd1400c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13ed9d640 1 -- 172.21.15.66:0/1111118837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1400cfbc0 msgr2=0x7fd14015a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13ed9d640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1400cfbc0 0x7fd14015a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13ed9d640 1 -- 172.21.15.66:0/1111118837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1400c9fb0 con 0x7fd1400d2d70 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.727+0000 7fd13ed9d640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1400d2d70 0x7fd1400b5560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd13800ebf0 tx=0x7fd13800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.728+0000 7fd136ffd640 1 -- 172.21.15.66:0/1111118837 <== mon.1 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138003b70 con 0x7fd1400d2d70 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.728+0000 7fd136ffd640 1 -- 172.21.15.66:0/1111118837 <== mon.1 v2:172.21.15.94:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd138003d10 con 0x7fd1400d2d70 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.728+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1400ca290 con 0x7fd1400d2d70 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.728+0000 7fd136ffd640 1 -- 172.21.15.66:0/1111118837 <== mon.1 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138004280 con 0x7fd1400d2d70 2024-09-10T22:38:39.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.728+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1400ca850 con 0x7fd1400d2d70 2024-09-10T22:38:39.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.729+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1400cf680 con 0x7fd1400d2d70 2024-09-10T22:38:39.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.729+0000 7fd136ffd640 1 -- 172.21.15.66:0/1111118837 <== mon.1 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd138004420 con 0x7fd1400d2d70 2024-09-10T22:38:39.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.730+0000 7fd136ffd640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd120077f70 0x7fd12007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:39.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.730+0000 7fd136ffd640 1 -- 172.21.15.66:0/1111118837 <== mon.1 v2:172.21.15.94:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5609+0+0 (secure 0 0 0) 0x7fd13809a950 con 0x7fd1400d2d70 2024-09-10T22:38:39.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.730+0000 7fd13e59c640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd120077f70 0x7fd12007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:39.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.731+0000 7fd13e59c640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd120077f70 0x7fd12007a430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd130000c00 tx=0x7fd130004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:39.736 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.735+0000 7fd136ffd640 1 -- 172.21.15.66:0/1111118837 <== mon.1 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd138062dc0 con 0x7fd1400d2d70 2024-09-10T22:38:39.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:39.957+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd1400d3e40 con 0x7fd1400d2d70 2024-09-10T22:38:40.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.590+0000 7fd136ffd640 1 -- 172.21.15.66:0/1111118837 <== mon.1 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v29) ==== 172+0+0 (secure 0 0 0) 0x7fd1380677f0 con 0x7fd1400d2d70 2024-09-10T22:38:40.592 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T22:38:40.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.607+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd120077f70 msgr2=0x7fd12007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:40.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.607+0000 7fd144c9b640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd120077f70 0x7fd12007a430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd130000c00 tx=0x7fd130004730 comp rx=0 tx=0).stop 2024-09-10T22:38:40.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.607+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1400d2d70 msgr2=0x7fd1400b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:40.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.607+0000 7fd144c9b640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1400d2d70 0x7fd1400b5560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd13800ebf0 tx=0x7fd13800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 shutdown_connections 2024-09-10T22:38:40.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd120077f70 0x7fd12007a430 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd1400d2d70 0x7fd1400b5560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd1400d07a0 0x7fd1400c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 --2- 172.21.15.66:0/1111118837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1400cfbc0 0x7fd14015a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 >> 172.21.15.66:0/1111118837 conn(0x7fd1400d4060 msgr2=0x7fd1400b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:40.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 shutdown_connections 2024-09-10T22:38:40.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.610+0000 7fd144c9b640 1 -- 172.21.15.66:0/1111118837 wait complete. 2024-09-10T22:38:40.630 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.735+0000 7eff7bd52640 1 -- 172.21.15.66:0/1753308879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff600051c0 msgr2=0x7eff6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.735+0000 7eff7bd52640 1 --2- 172.21.15.66:0/1753308879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff600051c0 0x7eff6000b9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7eff740b18d0 tx=0x7eff60005830 comp rx=0 tx=0).stop 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.735+0000 7eff7bd52640 1 -- 172.21.15.66:0/1753308879 shutdown_connections 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.735+0000 7eff7bd52640 1 --2- 172.21.15.66:0/1753308879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff600051c0 0x7eff6000b9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.735+0000 7eff7bd52640 1 -- 172.21.15.66:0/1753308879 >> 172.21.15.66:0/1753308879 conn(0x7eff74074290 msgr2=0x7eff740746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.735+0000 7eff7bd52640 1 -- 172.21.15.66:0/1753308879 shutdown_connections 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.735+0000 7eff7bd52640 1 -- 172.21.15.66:0/1753308879 wait complete. 2024-09-10T22:38:40.739 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 Processor -- start 2024-09-10T22:38:40.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 -- start start 2024-09-10T22:38:40.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff600051c0 0x7eff741f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7415ba60 0x7eff7415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff740b5930 0x7eff740b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7eff740ce740 con 0x7eff7415ba60 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff740ce5c0 con 0x7eff740b5930 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff7bd52640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7eff740ccdd0 con 0x7eff600051c0 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.737+0000 7eff792ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7415ba60 0x7eff7415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff79aee640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff600051c0 0x7eff741f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7415ba60 0x7eff7415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38014/0 (socket says 172.21.15.66:38014) 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 -- 172.21.15.66:0/2437330112 learned_addr learned my addr 172.21.15.66:0/2437330112 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff7a2ef640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff740b5930 0x7eff740b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 -- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff600051c0 msgr2=0x7eff741f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff600051c0 0x7eff741f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 -- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff740b5930 msgr2=0x7eff740b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff740b5930 0x7eff740b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 -- 172.21.15.66:0/2437330112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff74099ef0 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff792ed640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7415ba60 0x7eff7415bee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7eff68011900 tx=0x7eff68011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff667fc640 1 -- 172.21.15.66:0/2437330112 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6800fe70 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff667fc640 1 -- 172.21.15.66:0/2437330112 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff68004050 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7415a200 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff667fc640 1 -- 172.21.15.66:0/2437330112 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff680044a0 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.738+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7415a7c0 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.739+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff740b0430 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.740+0000 7eff667fc640 1 -- 172.21.15.66:0/2437330112 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7eff68018070 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.741+0000 7eff667fc640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7eff58077fa0 0x7eff5807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.741+0000 7eff667fc640 1 -- 172.21.15.66:0/2437330112 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7eff6809ada0 con 0x7eff7415ba60 2024-09-10T22:38:40.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.741+0000 7eff79aee640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7eff58077fa0 0x7eff5807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:40.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.741+0000 7eff79aee640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7eff58077fa0 0x7eff5807a460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:40.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.744+0000 7eff667fc640 1 -- 172.21.15.66:0/2437330112 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7eff74099ef0 con 0x7eff7415ba60 2024-09-10T22:38:40.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.930+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff740cb120 con 0x7eff7415ba60 2024-09-10T22:38:40.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.931+0000 7eff667fc640 1 -- 172.21.15.66:0/2437330112 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16662 (secure 0 0 0) 0x7eff68063330 con 0x7eff7415ba60 2024-09-10T22:38:40.933 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:40.934 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":46,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:38:39.959465+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:38:37.601238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:38:38.924362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:38:40.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7eff58077fa0 msgr2=0x7eff5807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:40.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7eff58077fa0 0x7eff5807a460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7eff70006fd0 tx=0x7eff70006b10 comp rx=0 tx=0).stop 2024-09-10T22:38:40.935 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7415ba60 msgr2=0x7eff7415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7415ba60 0x7eff7415bee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7eff68011900 tx=0x7eff68011dc0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 shutdown_connections 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7eff58077fa0 0x7eff5807a460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff740b5930 0x7eff740b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff7415ba60 0x7eff7415bee0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 --2- 172.21.15.66:0/2437330112 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7eff600051c0 0x7eff741f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 >> 172.21.15.66:0/2437330112 conn(0x7eff74074290 msgr2=0x7eff740c6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 shutdown_connections 2024-09-10T22:38:40.936 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:40.933+0000 7eff7bd52640 1 -- 172.21.15.66:0/2437330112 wait complete. 2024-09-10T22:38:40.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:38:41.053 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 -- 172.21.15.66:0/4222017517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80cdb60 msgr2=0x7f84f815f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/4222017517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80cdb60 0x7f84f815f000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84f0005fd0 tx=0x7f84f002a940 comp rx=0 tx=0).stop 2024-09-10T22:38:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 -- 172.21.15.66:0/4222017517 shutdown_connections 2024-09-10T22:38:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/4222017517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80cdb60 0x7f84f815f000 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/4222017517 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84f80ce830 0x7f84f80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 -- 172.21.15.66:0/4222017517 >> 172.21.15.66:0/4222017517 conn(0x7f84f80d1e30 msgr2=0x7f84f80d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:41.054 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 -- 172.21.15.66:0/4222017517 shutdown_connections 2024-09-10T22:38:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.050+0000 7f84fdf1f640 1 -- 172.21.15.66:0/4222017517 wait complete. 2024-09-10T22:38:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.051+0000 7f84fdf1f640 1 Processor -- start 2024-09-10T22:38:41.055 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.051+0000 7f84fdf1f640 1 -- start start 2024-09-10T22:38:41.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84fdf1f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f80cdb60 0x7f84f8159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84fdf1f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84f80ce830 0x7f84f81f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84fdf1f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80710f0 0x7f84f8071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84fdf1f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80d2b10 con 0x7f84f80cdb60 2024-09-10T22:38:41.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84fdf1f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80d2990 con 0x7f84f80710f0 2024-09-10T22:38:41.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84fdf1f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f84f80cec10 con 0x7f84f80ce830 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f80cdb60 0x7f84f8159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f80cdb60 0x7f84f8159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38030/0 (socket says 172.21.15.66:38030) 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 -- 172.21.15.66:0/2099029101 learned_addr learned my addr 172.21.15.66:0/2099029101 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f7fff640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80710f0 0x7f84f8071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84effff640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84f80ce830 0x7f84f81f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 -- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84f80ce830 msgr2=0x7f84f81f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84f80ce830 0x7f84f81f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 -- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80710f0 msgr2=0x7f84f8071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80710f0 0x7f84f8071560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 -- 172.21.15.66:0/2099029101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f81f3be0 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.052+0000 7f84f77fe640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f80cdb60 0x7f84f8159f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f84e800e9c0 tx=0x7f84e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.053+0000 7f84f4ff9640 1 -- 172.21.15.66:0/2099029101 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e800cd80 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.053+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f81f3ec0 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.053+0000 7f84f4ff9640 1 -- 172.21.15.66:0/2099029101 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84e8004020 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.053+0000 7f84f4ff9640 1 -- 172.21.15.66:0/2099029101 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8004450 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.053+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f815a480 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.054+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f80b4c80 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.054+0000 7f84f4ff9640 1 -- 172.21.15.66:0/2099029101 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f84e80035f0 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.055+0000 7f84f4ff9640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f84d8077f70 0x7f84d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.055+0000 7f84f4ff9640 1 -- 172.21.15.66:0/2099029101 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f84e809a590 con 0x7f84f80cdb60 2024-09-10T22:38:41.062 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.055+0000 7f84effff640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f84d8077f70 0x7f84d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.055+0000 7f84effff640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f84d8077f70 0x7f84d807a430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f84f809f730 tx=0x7f84e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:41.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.059+0000 7f84f4ff9640 1 -- 172.21.15.66:0/2099029101 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f84e8062a50 con 0x7f84f80cdb60 2024-09-10T22:38:41.248 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.246+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84f80d1160 con 0x7f84f80cdb60 2024-09-10T22:38:41.249 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.247+0000 7f84f4ff9640 1 -- 172.21.15.66:0/2099029101 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16662 (secure 0 0 0) 0x7f84e8067480 con 0x7f84f80cdb60 2024-09-10T22:38:41.250 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:41.250 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":46,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:38:39.959465+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:38:37.601238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:38:38.924362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:38:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f84d8077f70 msgr2=0x7f84d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f84d8077f70 0x7f84d807a430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f84f809f730 tx=0x7f84e0009040 comp rx=0 tx=0).stop 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f80cdb60 msgr2=0x7f84f8159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f80cdb60 0x7f84f8159f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f84e800e9c0 tx=0x7f84e800ee80 comp rx=0 tx=0).stop 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 shutdown_connections 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f84d8077f70 0x7f84d807a430 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f80710f0 0x7f84f8071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f84f80ce830 0x7f84f81f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 --2- 172.21.15.66:0/2099029101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f80cdb60 0x7f84f8159f40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 >> 172.21.15.66:0/2099029101 conn(0x7f84f80d1e30 msgr2=0x7f84f80b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 shutdown_connections 2024-09-10T22:38:41.253 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.250+0000 7f84fdf1f640 1 -- 172.21.15.66:0/2099029101 wait complete. 2024-09-10T22:38:41.263 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:38:41.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 -- 172.21.15.66:0/2919068687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f557406f810 msgr2=0x7f5574164290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 --2- 172.21.15.66:0/2919068687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f557406f810 0x7f5574164290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f556c005fd0 tx=0x7f556c02aa10 comp rx=0 tx=0).stop 2024-09-10T22:38:41.372 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 -- 172.21.15.66:0/2919068687 shutdown_connections 2024-09-10T22:38:41.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 --2- 172.21.15.66:0/2919068687 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f557406f810 0x7f5574164290 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 --2- 172.21.15.66:0/2919068687 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f557415b430 0x7f55740ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 -- 172.21.15.66:0/2919068687 >> 172.21.15.66:0/2919068687 conn(0x7f55740b1950 msgr2=0x7f55740b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:41.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 -- 172.21.15.66:0/2919068687 shutdown_connections 2024-09-10T22:38:41.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.370+0000 7f557a375640 1 -- 172.21.15.66:0/2919068687 wait complete. 2024-09-10T22:38:41.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.371+0000 7f557a375640 1 Processor -- start 2024-09-10T22:38:41.373 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.371+0000 7f557a375640 1 -- start start 2024-09-10T22:38:41.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f557a375640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557406f810 0x7f55740c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f557a375640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55740ca9f0 0x7f55740c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f557a375640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f557415b430 0x7f55740cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.375 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f557a375640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5574164950 con 0x7f55740ca9f0 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f557a375640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f55741647d0 con 0x7f557406f810 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f557a375640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5574164ad0 con 0x7f557415b430 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55740ca9f0 0x7f55740c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55740ca9f0 0x7f55740c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38050/0 (socket says 172.21.15.66:38050) 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 -- 172.21.15.66:0/989625640 learned_addr learned my addr 172.21.15.66:0/989625640 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f5573fff640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557406f810 0x7f55740c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f5578912640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f557415b430 0x7f55740cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 -- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f557415b430 msgr2=0x7f55740cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f557415b430 0x7f55740cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 -- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557406f810 msgr2=0x7f55740c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557406f810 0x7f55740c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.372+0000 7f55737fe640 1 -- 172.21.15.66:0/989625640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55740b2e30 con 0x7f55740ca9f0 2024-09-10T22:38:41.376 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.373+0000 7f55737fe640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55740ca9f0 0x7f55740c9900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f556800c480 tx=0x7f556800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:41.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.373+0000 7f5570ff9640 1 -- 172.21.15.66:0/989625640 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55680052d0 con 0x7f55740ca9f0 2024-09-10T22:38:41.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.373+0000 7f5570ff9640 1 -- 172.21.15.66:0/989625640 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5568005470 con 0x7f55740ca9f0 2024-09-10T22:38:41.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.373+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55740b3e00 con 0x7f55740ca9f0 2024-09-10T22:38:41.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.373+0000 7f5570ff9640 1 -- 172.21.15.66:0/989625640 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568012870 con 0x7f55740ca9f0 2024-09-10T22:38:41.377 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.373+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55740b4600 con 0x7f55740ca9f0 2024-09-10T22:38:41.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.374+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55740ca4b0 con 0x7f55740ca9f0 2024-09-10T22:38:41.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.375+0000 7f5570ff9640 1 -- 172.21.15.66:0/989625640 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f55680035f0 con 0x7f55740ca9f0 2024-09-10T22:38:41.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.375+0000 7f5570ff9640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5560077fa0 0x7f556007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:41.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.375+0000 7f5570ff9640 1 -- 172.21.15.66:0/989625640 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f55680997a0 con 0x7f55740ca9f0 2024-09-10T22:38:41.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.375+0000 7f5573fff640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5560077fa0 0x7f556007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:41.378 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.376+0000 7f5573fff640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5560077fa0 0x7f556007a460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55741f5e90 tx=0x7f5564005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:41.381 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.380+0000 7f5570ff9640 1 -- 172.21.15.66:0/989625640 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5568061ce0 con 0x7f55740ca9f0 2024-09-10T22:38:41.606 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:41.604+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f55740b27b0 con 0x7f55740ca9f0 2024-09-10T22:38:42.573 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:38:42.571+0000 7effc4a01640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:38:42.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.606+0000 7f5570ff9640 1 -- 172.21.15.66:0/989625640 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7f5568066710 con 0x7f55740ca9f0 2024-09-10T22:38:42.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5560077fa0 msgr2=0x7f556007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:42.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5560077fa0 0x7f556007a460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55741f5e90 tx=0x7f5564005f90 comp rx=0 tx=0).stop 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55740ca9f0 msgr2=0x7f55740c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55740ca9f0 0x7f55740c9900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f556800c480 tx=0x7f556800c940 comp rx=0 tx=0).stop 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 shutdown_connections 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5560077fa0 0x7f556007a460 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f557415b430 0x7f55740cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55740ca9f0 0x7f55740c9900 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 --2- 172.21.15.66:0/989625640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f557406f810 0x7f55740c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 >> 172.21.15.66:0/989625640 conn(0x7f55740b1950 msgr2=0x7f55740cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 shutdown_connections 2024-09-10T22:38:42.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.610+0000 7f557a375640 1 -- 172.21.15.66:0/989625640 wait complete. 2024-09-10T22:38:42.626 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T22:38:42.626 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T22:38:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.739+0000 7f9e27f84640 1 -- 172.21.15.66:0/316601898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200cbe90 msgr2=0x7f9e200b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.739+0000 7f9e27f84640 1 --2- 172.21.15.66:0/316601898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200cbe90 0x7f9e200b1130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9e20073370 tx=0x7f9e1002a510 comp rx=0 tx=0).stop 2024-09-10T22:38:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.739+0000 7f9e27f84640 1 -- 172.21.15.66:0/316601898 shutdown_connections 2024-09-10T22:38:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.739+0000 7f9e27f84640 1 --2- 172.21.15.66:0/316601898 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200cbe90 0x7f9e200b1130 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.739+0000 7f9e27f84640 1 -- 172.21.15.66:0/316601898 >> 172.21.15.66:0/316601898 conn(0x7f9e20074290 msgr2=0x7f9e200746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.739+0000 7f9e27f84640 1 -- 172.21.15.66:0/316601898 shutdown_connections 2024-09-10T22:38:42.741 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.739+0000 7f9e27f84640 1 -- 172.21.15.66:0/316601898 wait complete. 2024-09-10T22:38:42.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.740+0000 7f9e27f84640 1 Processor -- start 2024-09-10T22:38:42.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.740+0000 7f9e27f84640 1 -- start start 2024-09-10T22:38:42.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e27f84640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e200cbe90 0x7f9e201f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:42.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e27f84640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e200cd950 0x7f9e200ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:42.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e27f84640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200d0ee0 0x7f9e201eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:42.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e27f84640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f9e200b0680 con 0x7f9e200d0ee0 2024-09-10T22:38:42.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e27f84640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e200b0500 con 0x7f9e200cd950 2024-09-10T22:38:42.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e27f84640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f9e200b0800 con 0x7f9e200cbe90 2024-09-10T22:38:42.744 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200d0ee0 0x7f9e201eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200d0ee0 0x7f9e201eb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38076/0 (socket says 172.21.15.66:38076) 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 -- 172.21.15.66:0/3365911199 learned_addr learned my addr 172.21.15.66:0/3365911199 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e2551f640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e200cd950 0x7f9e200ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e25d20640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e200cbe90 0x7f9e201f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 -- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e200cbe90 msgr2=0x7f9e201f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e200cbe90 0x7f9e201f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 -- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e200cd950 msgr2=0x7f9e200ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e200cd950 0x7f9e200ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 -- 172.21.15.66:0/3365911199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e200ce5c0 con 0x7f9e200d0ee0 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e25d20640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e200cbe90 0x7f9e201f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.741+0000 7f9e26521640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200d0ee0 0x7f9e201eb8b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800cae0 tx=0x7f9e1800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.742+0000 7f9e167fc640 1 -- 172.21.15.66:0/3365911199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18016020 con 0x7f9e200d0ee0 2024-09-10T22:38:42.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.742+0000 7f9e167fc640 1 -- 172.21.15.66:0/3365911199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e1800ad30 con 0x7f9e200d0ee0 2024-09-10T22:38:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.742+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e201ebdf0 con 0x7f9e200d0ee0 2024-09-10T22:38:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.742+0000 7f9e167fc640 1 -- 172.21.15.66:0/3365911199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18012680 con 0x7f9e200d0ee0 2024-09-10T22:38:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.742+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e200ceb50 con 0x7f9e200d0ee0 2024-09-10T22:38:42.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.743+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e20156830 con 0x7f9e200d0ee0 2024-09-10T22:38:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.743+0000 7f9e167fc640 1 -- 172.21.15.66:0/3365911199 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9e1801b400 con 0x7f9e200d0ee0 2024-09-10T22:38:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.744+0000 7f9e167fc640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9e04077f70 0x7f9e0407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.744+0000 7f9e167fc640 1 -- 172.21.15.66:0/3365911199 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f9e1809a650 con 0x7f9e200d0ee0 2024-09-10T22:38:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.744+0000 7f9e25d20640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9e04077f70 0x7f9e0407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:42.747 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.745+0000 7f9e25d20640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9e04077f70 0x7f9e0407a430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c001750 tx=0x7f9e1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:42.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.749+0000 7f9e167fc640 1 -- 172.21.15.66:0/3365911199 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9e20156830 con 0x7f9e200d0ee0 2024-09-10T22:38:42.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:42.974+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9e2015c150 con 0x7f9e200d0ee0 2024-09-10T22:38:43.618 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.616+0000 7f9e167fc640 1 -- 172.21.15.66:0/3365911199 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f9e18020020 con 0x7f9e200d0ee0 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9e04077f70 msgr2=0x7f9e0407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9e04077f70 0x7f9e0407a430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c001750 tx=0x7f9e1c009040 comp rx=0 tx=0).stop 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200d0ee0 msgr2=0x7f9e201eb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200d0ee0 0x7f9e201eb8b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800cae0 tx=0x7f9e1800a060 comp rx=0 tx=0).stop 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 shutdown_connections 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9e04077f70 0x7f9e0407a430 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9e200d0ee0 0x7f9e201eb8b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e200cd950 0x7f9e200ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 --2- 172.21.15.66:0/3365911199 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f9e200cbe90 0x7f9e201f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.619+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 >> 172.21.15.66:0/3365911199 conn(0x7f9e20074290 msgr2=0x7f9e200ad780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.620+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 shutdown_connections 2024-09-10T22:38:43.622 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.620+0000 7f9e27f84640 1 -- 172.21.15.66:0/3365911199 wait complete. 2024-09-10T22:38:43.634 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T22:38:43.635 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:43.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.742+0000 7f0eedc71640 1 -- 172.21.15.66:0/2829255509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80d2d70 msgr2=0x7f0ee80d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:43.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.742+0000 7f0eedc71640 1 --2- 172.21.15.66:0/2829255509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80d2d70 0x7f0ee80d0b80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80b5af0 tx=0x7f0edc02a560 comp rx=0 tx=0).stop 2024-09-10T22:38:43.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.743+0000 7f0eedc71640 1 -- 172.21.15.66:0/2829255509 shutdown_connections 2024-09-10T22:38:43.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.743+0000 7f0eedc71640 1 --2- 172.21.15.66:0/2829255509 >> v2:172.21.15.94:3300/0 conn(0x7f0ee80cfbc0 0x7f0ee80d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.743+0000 7f0eedc71640 1 --2- 172.21.15.66:0/2829255509 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80d2d70 0x7f0ee80d0b80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.745 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.743+0000 7f0eedc71640 1 --2- 172.21.15.66:0/2829255509 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ee80d07a0 0x7f0ee80cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.743+0000 7f0eedc71640 1 -- 172.21.15.66:0/2829255509 >> 172.21.15.66:0/2829255509 conn(0x7f0ee80d4060 msgr2=0x7f0ee80d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:43.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.743+0000 7f0eedc71640 1 -- 172.21.15.66:0/2829255509 shutdown_connections 2024-09-10T22:38:43.746 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.743+0000 7f0eedc71640 1 -- 172.21.15.66:0/2829255509 wait complete. 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.744+0000 7f0eedc71640 1 Processor -- start 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.744+0000 7f0eedc71640 1 -- start start 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0eedc71640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80cfbc0 0x7f0ee81f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0eedc71640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee80d07a0 0x7f0ee80c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0eedc71640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ee80d2d70 0x7f0ee80cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0eedc71640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee80b3e30 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0eedc71640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee80b3cb0 con 0x7f0ee80d07a0 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0eedc71640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee80b3fb0 con 0x7f0ee80d2d70 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80cfbc0 0x7f0ee81f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80cfbc0 0x7f0ee81f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48544/0 (socket says 172.21.15.66:48544) 2024-09-10T22:38:43.748 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 -- 172.21.15.66:0/9340161 learned_addr learned my addr 172.21.15.66:0/9340161 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee6ffd640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee80d07a0 0x7f0ee80c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee7fff640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ee80d2d70 0x7f0ee80cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 -- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ee80d2d70 msgr2=0x7f0ee80cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ee80d2d70 0x7f0ee80cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 -- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee80d07a0 msgr2=0x7f0ee80c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee80d07a0 0x7f0ee80c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 -- 172.21.15.66:0/9340161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee8156790 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ee77fe640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80cfbc0 0x7f0ee81f4c70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800ebf0 tx=0x7f0ed800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.745+0000 7f0ecffff640 1 -- 172.21.15.66:0/9340161 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8010630 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.749 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.746+0000 7f0ecffff640 1 -- 172.21.15.66:0/9340161 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ed80108c0 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.746+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee81f53c0 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.746+0000 7f0ecffff640 1 -- 172.21.15.66:0/9340161 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8010d10 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.750 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.746+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee81f5980 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.747+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee80cf680 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.747+0000 7f0ecffff640 1 -- 172.21.15.66:0/9340161 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0ed8018420 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.748+0000 7f0ecffff640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ed4077f70 0x7f0ed407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:43.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.748+0000 7f0ecffff640 1 -- 172.21.15.66:0/9340161 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f0ed809aab0 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.748+0000 7f0ee6ffd640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ed4077f70 0x7f0ed407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:43.751 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.749+0000 7f0ee6ffd640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ed4077f70 0x7f0ed407a430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0edc033000 tx=0x7f0edc004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:43.754 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.752+0000 7f0ecffff640 1 -- 172.21.15.66:0/9340161 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0ed8062ff0 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.979 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.977+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ee80d3e40 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.978+0000 7f0ecffff640 1 -- 172.21.15.66:0/9340161 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5223 (secure 0 0 0) 0x7f0ed8067a20 con 0x7f0ee80cfbc0 2024-09-10T22:38:43.980 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:43.980 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":32,"btime":"2024-09-10T22:38:43:613077+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3548861128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3548861128},{"type":"v1","addr":"172.21.15.94:6837","nonce":3548861128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1690198522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1690198522},{"type":"v1","addr":"172.21.15.94:6836","nonce":1690198522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1833881771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1833881771},{"type":"v1","addr":"172.21.15.66:6834","nonce":1833881771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:42.976687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3999038135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3999038135},{"type":"v1","addr":"172.21.15.66:6835","nonce":3999038135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5304,"qdb_cluster":[5304]},"id":3}]} 2024-09-10T22:38:43.980 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 32 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ed4077f70 msgr2=0x7f0ed407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ed4077f70 0x7f0ed407a430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0edc033000 tx=0x7f0edc004730 comp rx=0 tx=0).stop 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80cfbc0 msgr2=0x7f0ee81f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80cfbc0 0x7f0ee81f4c70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800ebf0 tx=0x7f0ed800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 shutdown_connections 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0ed4077f70 0x7f0ed407a430 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0ee80d2d70 0x7f0ee80cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee80d07a0 0x7f0ee80c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 --2- 172.21.15.66:0/9340161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ee80cfbc0 0x7f0ee81f4c70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 >> 172.21.15.66:0/9340161 conn(0x7f0ee80d4060 msgr2=0x7f0ee80b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 shutdown_connections 2024-09-10T22:38:43.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:43.980+0000 7f0eedc71640 1 -- 172.21.15.66:0/9340161 wait complete. 2024-09-10T22:38:43.993 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 -- 172.21.15.66:0/1239767878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff974151a90 msgr2=0x7ff97415bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/1239767878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff974151a90 0x7ff97415bc80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005fd0 tx=0x7ff96c02a5d0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 -- 172.21.15.66:0/1239767878 shutdown_connections 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/1239767878 >> v2:172.21.15.94:3300/0 conn(0x7ff9740ccee0 0x7ff974073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/1239767878 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff974151a90 0x7ff97415bc80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 -- 172.21.15.66:0/1239767878 >> 172.21.15.66:0/1239767878 conn(0x7ff9740b1d50 msgr2=0x7ff9740b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 -- 172.21.15.66:0/1239767878 shutdown_connections 2024-09-10T22:38:44.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.096+0000 7ff97bbd3640 1 -- 172.21.15.66:0/1239767878 wait complete. 2024-09-10T22:38:44.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 Processor -- start 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 -- start start 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9740caa70 0x7ff9740b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9740ccee0 0x7ff97413eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974151a90 0x7ff9741579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff974073f50 con 0x7ff974151a90 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff974073dd0 con 0x7ff9740ccee0 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97bbd3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff9740740d0 con 0x7ff9740caa70 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.098+0000 7ff97a170640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974151a90 0x7ff9741579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974151a90 0x7ff9741579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48552/0 (socket says 172.21.15.66:48552) 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 -- 172.21.15.66:0/2844917914 learned_addr learned my addr 172.21.15.66:0/2844917914 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97996f640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9740caa70 0x7ff9740b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.102 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97916e640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9740ccee0 0x7ff97413eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 -- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9740caa70 msgr2=0x7ff9740b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9740caa70 0x7ff9740b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 -- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9740ccee0 msgr2=0x7ff97413eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9740ccee0 0x7ff97413eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 -- 172.21.15.66:0/2844917914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9741e5f60 con 0x7ff974151a90 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97a170640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974151a90 0x7ff9741579c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff96400eb40 tx=0x7ff96400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff9727fc640 1 -- 172.21.15.66:0/2844917914 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964004b20 con 0x7ff974151a90 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff9727fc640 1 -- 172.21.15.66:0/2844917914 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff964004db0 con 0x7ff974151a90 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9741e6240 con 0x7ff974151a90 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff9727fc640 1 -- 172.21.15.66:0/2844917914 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964005280 con 0x7ff974151a90 2024-09-10T22:38:44.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.099+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9741f6d20 con 0x7ff974151a90 2024-09-10T22:38:44.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.100+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff974074580 con 0x7ff974151a90 2024-09-10T22:38:44.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.101+0000 7ff9727fc640 1 -- 172.21.15.66:0/2844917914 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff964005420 con 0x7ff974151a90 2024-09-10T22:38:44.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.102+0000 7ff9727fc640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff960077f70 0x7ff96007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.102+0000 7ff9727fc640 1 -- 172.21.15.66:0/2844917914 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7ff96409aad0 con 0x7ff974151a90 2024-09-10T22:38:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.102+0000 7ff97996f640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff960077f70 0x7ff96007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.105 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.102+0000 7ff97996f640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff960077f70 0x7ff96007a430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff96c033000 tx=0x7ff96c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:44.108 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.106+0000 7ff9727fc640 1 -- 172.21.15.66:0/2844917914 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff964062f90 con 0x7ff974151a90 2024-09-10T22:38:44.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.292+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9741e34a0 con 0x7ff974151a90 2024-09-10T22:38:44.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.293+0000 7ff9727fc640 1 -- 172.21.15.66:0/2844917914 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16649 (secure 0 0 0) 0x7ff9640679c0 con 0x7ff974151a90 2024-09-10T22:38:44.295 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:44.295 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":47,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:38:41.586360+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:38:37.601238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:38:38.924362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:38:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff960077f70 msgr2=0x7ff96007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff960077f70 0x7ff96007a430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff96c033000 tx=0x7ff96c004730 comp rx=0 tx=0).stop 2024-09-10T22:38:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974151a90 msgr2=0x7ff9741579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974151a90 0x7ff9741579c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff96400eb40 tx=0x7ff96400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 shutdown_connections 2024-09-10T22:38:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff960077f70 0x7ff96007a430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff974151a90 0x7ff9741579c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9740ccee0 0x7ff97413eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 --2- 172.21.15.66:0/2844917914 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff9740caa70 0x7ff9740b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 >> 172.21.15.66:0/2844917914 conn(0x7ff9740b1d50 msgr2=0x7ff97414f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 shutdown_connections 2024-09-10T22:38:44.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.295+0000 7ff97bbd3640 1 -- 172.21.15.66:0/2844917914 wait complete. 2024-09-10T22:38:44.308 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:38:44.308 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:38:44.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 -- 172.21.15.66:0/2488456632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c152a40 msgr2=0x7f811c152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 --2- 172.21.15.66:0/2488456632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c152a40 0x7f811c152e20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f811c0cc410 tx=0x7f810c02a4d0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 -- 172.21.15.66:0/2488456632 shutdown_connections 2024-09-10T22:38:44.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 --2- 172.21.15.66:0/2488456632 >> v2:172.21.15.94:3300/0 conn(0x7f811c072810 0x7f811c15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 --2- 172.21.15.66:0/2488456632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c152a40 0x7f811c152e20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 -- 172.21.15.66:0/2488456632 >> 172.21.15.66:0/2488456632 conn(0x7f811c0b1950 msgr2=0x7f811c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:44.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 -- 172.21.15.66:0/2488456632 shutdown_connections 2024-09-10T22:38:44.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.412+0000 7f812126a640 1 -- 172.21.15.66:0/2488456632 wait complete. 2024-09-10T22:38:44.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.413+0000 7f812126a640 1 Processor -- start 2024-09-10T22:38:44.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.413+0000 7f812126a640 1 -- start start 2024-09-10T22:38:44.417 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f812126a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c072810 0x7f811c14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f812126a640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c0ca4b0 0x7f811c0c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f812126a640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c152a40 0x7f811c0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f812126a640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f811c0caa30 con 0x7f811c0ca4b0 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f812126a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f811c0ca8b0 con 0x7f811c072810 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f812126a640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f811c0cabb0 con 0x7f811c152a40 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c0ca4b0 0x7f811c0c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c0ca4b0 0x7f811c0c3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48562/0 (socket says 172.21.15.66:48562) 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 -- 172.21.15.66:0/1459250760 learned_addr learned my addr 172.21.15.66:0/1459250760 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f811affd640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c072810 0x7f811c14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f811b7fe640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c152a40 0x7f811c0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 -- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c152a40 msgr2=0x7f811c0c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c152a40 0x7f811c0c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 -- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c072810 msgr2=0x7f811c14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c072810 0x7f811c14a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 -- 172.21.15.66:0/1459250760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c0c4370 con 0x7f811c0ca4b0 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f811b7fe640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c152a40 0x7f811c0c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.414+0000 7f8113fff640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c0ca4b0 0x7f811c0c3990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f810400d950 tx=0x7f810400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.415+0000 7f81137fe640 1 -- 172.21.15.66:0/1459250760 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104017070 con 0x7f811c0ca4b0 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.415+0000 7f81137fe640 1 -- 172.21.15.66:0/1459250760 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8104009d20 con 0x7f811c0ca4b0 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.415+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811c0c4650 con 0x7f811c0ca4b0 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.415+0000 7f81137fe640 1 -- 172.21.15.66:0/1459250760 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040106c0 con 0x7f811c0ca4b0 2024-09-10T22:38:44.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.415+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811c153770 con 0x7f811c0ca4b0 2024-09-10T22:38:44.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.416+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f811c15c2d0 con 0x7f811c0ca4b0 2024-09-10T22:38:44.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.416+0000 7f81137fe640 1 -- 172.21.15.66:0/1459250760 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8104010860 con 0x7f811c0ca4b0 2024-09-10T22:38:44.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.417+0000 7f81137fe640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f80fc077f70 0x7f80fc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.417+0000 7f81137fe640 1 -- 172.21.15.66:0/1459250760 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f810409a0d0 con 0x7f811c0ca4b0 2024-09-10T22:38:44.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.417+0000 7f811affd640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f80fc077f70 0x7f80fc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.418+0000 7f811affd640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f80fc077f70 0x7f80fc07a430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f810c033000 tx=0x7f810c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:44.423 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.421+0000 7f81137fe640 1 -- 172.21.15.66:0/1459250760 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8104062580 con 0x7f811c0ca4b0 2024-09-10T22:38:44.605 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.603+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f811c0b27d0 con 0x7f80fc077f70 2024-09-10T22:38:44.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.608+0000 7f81137fe640 1 -- 172.21.15.66:0/1459250760 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7f811c0b27d0 con 0x7f80fc077f70 2024-09-10T22:38:44.613 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:44.615 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f80fc077f70 msgr2=0x7f80fc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f80fc077f70 0x7f80fc07a430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f810c033000 tx=0x7f810c004730 comp rx=0 tx=0).stop 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c0ca4b0 msgr2=0x7f811c0c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c0ca4b0 0x7f811c0c3990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f810400d950 tx=0x7f810400de10 comp rx=0 tx=0).stop 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 shutdown_connections 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f80fc077f70 0x7f80fc07a430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f811c152a40 0x7f811c0c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f811c0ca4b0 0x7f811c0c3990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 --2- 172.21.15.66:0/1459250760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f811c072810 0x7f811c14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 >> 172.21.15.66:0/1459250760 conn(0x7f811c0b1950 msgr2=0x7f811c0b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 shutdown_connections 2024-09-10T22:38:44.617 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.612+0000 7f812126a640 1 -- 172.21.15.66:0/1459250760 wait complete. 2024-09-10T22:38:44.624 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-10T22:38:42.648505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242396,"kb_used_data":5316,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732324,"statfs":{"total":773094113280,"available":772845899776,"internally_reserved":0,"allocated":5443584,"data_stored":2896593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.067726"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.603728+0000","last_change":"2024-09-10T22:38:40.597900+0000","last_active":"2024-09-10T22:38:40.603728+0000","last_peered":"2024-09-10T22:38:40.603728+0000","last_clean":"2024-09-10T22:38:40.603728+0000","last_became_active":"2024-09-10T22:38:40.597608+0000","last_became_peered":"2024-09-10T22:38:40.597608+0000","last_unstale":"2024-09-10T22:38:40.603728+0000","last_undegraded":"2024-09-10T22:38:40.603728+0000","last_fullsized":"2024-09-10T22:38:40.603728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.090423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590792+0000","last_change":"2024-09-10T22:38:40.610451+0000","last_active":"2024-09-10T22:38:41.590792+0000","last_peered":"2024-09-10T22:38:41.590792+0000","last_clean":"2024-09-10T22:38:41.590792+0000","last_became_active":"2024-09-10T22:38:40.610162+0000","last_became_peered":"2024-09-10T22:38:40.610162+0000","last_unstale":"2024-09-10T22:38:41.590792+0000","last_undegraded":"2024-09-10T22:38:41.590792+0000","last_fullsized":"2024-09-10T22:38:41.590792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:12.308156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590708+0000","last_change":"2024-09-10T22:38:40.600360+0000","last_active":"2024-09-10T22:38:41.590708+0000","last_peered":"2024-09-10T22:38:41.590708+0000","last_clean":"2024-09-10T22:38:41.590708+0000","last_became_active":"2024-09-10T22:38:40.600094+0000","last_became_peered":"2024-09-10T22:38:40.600094+0000","last_unstale":"2024-09-10T22:38:41.590708+0000","last_undegraded":"2024-09-10T22:38:41.590708+0000","last_fullsized":"2024-09-10T22:38:41.590708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:48.453013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.615684+0000","last_change":"2024-09-10T22:38:40.612141+0000","last_active":"2024-09-10T22:38:40.615684+0000","last_peered":"2024-09-10T22:38:40.615684+0000","last_clean":"2024-09-10T22:38:40.615684+0000","last_became_active":"2024-09-10T22:38:40.609061+0000","last_became_peered":"2024-09-10T22:38:40.609061+0000","last_unstale":"2024-09-10T22:38:40.615684+0000","last_undegraded":"2024-09-10T22:38:40.615684+0000","last_fullsized":"2024-09-10T22:38:40.615684+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:11.983496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590703+0000","last_change":"2024-09-10T22:38:40.602735+0000","last_active":"2024-09-10T22:38:41.590703+0000","last_peered":"2024-09-10T22:38:41.590703+0000","last_clean":"2024-09-10T22:38:41.590703+0000","last_became_active":"2024-09-10T22:38:40.602387+0000","last_became_peered":"2024-09-10T22:38:40.602387+0000","last_unstale":"2024-09-10T22:38:41.590703+0000","last_undegraded":"2024-09-10T22:38:41.590703+0000","last_fullsized":"2024-09-10T22:38:41.590703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:14.101065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607645+0000","last_change":"2024-09-10T22:38:40.606772+0000","last_active":"2024-09-10T22:38:40.607645+0000","last_peered":"2024-09-10T22:38:40.607645+0000","last_clean":"2024-09-10T22:38:40.607645+0000","last_became_active":"2024-09-10T22:38:40.606525+0000","last_became_peered":"2024-09-10T22:38:40.606525+0000","last_unstale":"2024-09-10T22:38:40.607645+0000","last_undegraded":"2024-09-10T22:38:40.607645+0000","last_fullsized":"2024-09-10T22:38:40.607645+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:21.504845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.604879+0000","last_change":"2024-09-10T22:38:40.601289+0000","last_active":"2024-09-10T22:38:40.604879+0000","last_peered":"2024-09-10T22:38:40.604879+0000","last_clean":"2024-09-10T22:38:40.604879+0000","last_became_active":"2024-09-10T22:38:40.601024+0000","last_became_peered":"2024-09-10T22:38:40.601024+0000","last_unstale":"2024-09-10T22:38:40.604879+0000","last_undegraded":"2024-09-10T22:38:40.604879+0000","last_fullsized":"2024-09-10T22:38:40.604879+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:02.766716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616580+0000","last_change":"2024-09-10T22:38:40.612094+0000","last_active":"2024-09-10T22:38:40.616580+0000","last_peered":"2024-09-10T22:38:40.616580+0000","last_clean":"2024-09-10T22:38:40.616580+0000","last_became_active":"2024-09-10T22:38:40.609155+0000","last_became_peered":"2024-09-10T22:38:40.609155+0000","last_unstale":"2024-09-10T22:38:40.616580+0000","last_undegraded":"2024-09-10T22:38:40.616580+0000","last_fullsized":"2024-09-10T22:38:40.616580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:09.802304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.623209+0000","last_change":"2024-09-10T22:38:40.618587+0000","last_active":"2024-09-10T22:38:40.623209+0000","last_peered":"2024-09-10T22:38:40.623209+0000","last_clean":"2024-09-10T22:38:40.623209+0000","last_became_active":"2024-09-10T22:38:40.616718+0000","last_became_peered":"2024-09-10T22:38:40.616718+0000","last_unstale":"2024-09-10T22:38:40.623209+0000","last_undegraded":"2024-09-10T22:38:40.623209+0000","last_fullsized":"2024-09-10T22:38:40.623209+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:03.674615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616584+0000","last_change":"2024-09-10T22:38:40.612791+0000","last_active":"2024-09-10T22:38:40.616584+0000","last_peered":"2024-09-10T22:38:40.616584+0000","last_clean":"2024-09-10T22:38:40.616584+0000","last_became_active":"2024-09-10T22:38:40.612523+0000","last_became_peered":"2024-09-10T22:38:40.612523+0000","last_unstale":"2024-09-10T22:38:40.616584+0000","last_undegraded":"2024-09-10T22:38:40.616584+0000","last_fullsized":"2024-09-10T22:38:40.616584+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.241720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.615725+0000","last_change":"2024-09-10T22:38:40.606158+0000","last_active":"2024-09-10T22:38:40.615725+0000","last_peered":"2024-09-10T22:38:40.615725+0000","last_clean":"2024-09-10T22:38:40.615725+0000","last_became_active":"2024-09-10T22:38:40.605704+0000","last_became_peered":"2024-09-10T22:38:40.605704+0000","last_unstale":"2024-09-10T22:38:40.615725+0000","last_undegraded":"2024-09-10T22:38:40.615725+0000","last_fullsized":"2024-09-10T22:38:40.615725+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:07:11.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591653+0000","last_change":"2024-09-10T22:38:40.595559+0000","last_active":"2024-09-10T22:38:41.591653+0000","last_peered":"2024-09-10T22:38:41.591653+0000","last_clean":"2024-09-10T22:38:41.591653+0000","last_became_active":"2024-09-10T22:38:40.595295+0000","last_became_peered":"2024-09-10T22:38:40.595295+0000","last_unstale":"2024-09-10T22:38:41.591653+0000","last_undegraded":"2024-09-10T22:38:41.591653+0000","last_fullsized":"2024-09-10T22:38:41.591653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:57.734414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607841+0000","last_change":"2024-09-10T22:38:40.605923+0000","last_active":"2024-09-10T22:38:40.607841+0000","last_peered":"2024-09-10T22:38:40.607841+0000","last_clean":"2024-09-10T22:38:40.607841+0000","last_became_active":"2024-09-10T22:38:40.605695+0000","last_became_peered":"2024-09-10T22:38:40.605695+0000","last_unstale":"2024-09-10T22:38:40.607841+0000","last_undegraded":"2024-09-10T22:38:40.607841+0000","last_fullsized":"2024-09-10T22:38:40.607841+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:08.139735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616091+0000","last_change":"2024-09-10T22:38:40.608665+0000","last_active":"2024-09-10T22:38:40.616091+0000","last_peered":"2024-09-10T22:38:40.616091+0000","last_clean":"2024-09-10T22:38:40.616091+0000","last_became_active":"2024-09-10T22:38:40.608070+0000","last_became_peered":"2024-09-10T22:38:40.608070+0000","last_unstale":"2024-09-10T22:38:40.616091+0000","last_undegraded":"2024-09-10T22:38:40.616091+0000","last_fullsized":"2024-09-10T22:38:40.616091+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:03.518784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607726+0000","last_change":"2024-09-10T22:38:40.604613+0000","last_active":"2024-09-10T22:38:40.607726+0000","last_peered":"2024-09-10T22:38:40.607726+0000","last_clean":"2024-09-10T22:38:40.607726+0000","last_became_active":"2024-09-10T22:38:40.604264+0000","last_became_peered":"2024-09-10T22:38:40.604264+0000","last_unstale":"2024-09-10T22:38:40.607726+0000","last_undegraded":"2024-09-10T22:38:40.607726+0000","last_fullsized":"2024-09-10T22:38:40.607726+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:08.814068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.613699+0000","last_change":"2024-09-10T22:38:40.612699+0000","last_active":"2024-09-10T22:38:40.613699+0000","last_peered":"2024-09-10T22:38:40.613699+0000","last_clean":"2024-09-10T22:38:40.613699+0000","last_became_active":"2024-09-10T22:38:40.612229+0000","last_became_peered":"2024-09-10T22:38:40.612229+0000","last_unstale":"2024-09-10T22:38:40.613699+0000","last_undegraded":"2024-09-10T22:38:40.613699+0000","last_fullsized":"2024-09-10T22:38:40.613699+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:52.302762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591415+0000","last_change":"2024-09-10T22:38:40.594799+0000","last_active":"2024-09-10T22:38:41.591415+0000","last_peered":"2024-09-10T22:38:41.591415+0000","last_clean":"2024-09-10T22:38:41.591415+0000","last_became_active":"2024-09-10T22:38:40.593905+0000","last_became_peered":"2024-09-10T22:38:40.593905+0000","last_unstale":"2024-09-10T22:38:41.591415+0000","last_undegraded":"2024-09-10T22:38:41.591415+0000","last_fullsized":"2024-09-10T22:38:41.591415+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:46.906395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590974+0000","last_change":"2024-09-10T22:38:40.613412+0000","last_active":"2024-09-10T22:38:41.590974+0000","last_peered":"2024-09-10T22:38:41.590974+0000","last_clean":"2024-09-10T22:38:41.590974+0000","last_became_active":"2024-09-10T22:38:40.613122+0000","last_became_peered":"2024-09-10T22:38:40.613122+0000","last_unstale":"2024-09-10T22:38:41.590974+0000","last_undegraded":"2024-09-10T22:38:41.590974+0000","last_fullsized":"2024-09-10T22:38:41.590974+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:13.539196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.617741+0000","last_change":"2024-09-10T22:38:40.615868+0000","last_active":"2024-09-10T22:38:40.617741+0000","last_peered":"2024-09-10T22:38:40.617741+0000","last_clean":"2024-09-10T22:38:40.617741+0000","last_became_active":"2024-09-10T22:38:40.615487+0000","last_became_peered":"2024-09-10T22:38:40.615487+0000","last_unstale":"2024-09-10T22:38:40.617741+0000","last_undegraded":"2024-09-10T22:38:40.617741+0000","last_fullsized":"2024-09-10T22:38:40.617741+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:40.335621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.620627+0000","last_change":"2024-09-10T22:38:40.612550+0000","last_active":"2024-09-10T22:38:40.620627+0000","last_peered":"2024-09-10T22:38:40.620627+0000","last_clean":"2024-09-10T22:38:40.620627+0000","last_became_active":"2024-09-10T22:38:40.612245+0000","last_became_peered":"2024-09-10T22:38:40.612245+0000","last_unstale":"2024-09-10T22:38:40.620627+0000","last_undegraded":"2024-09-10T22:38:40.620627+0000","last_fullsized":"2024-09-10T22:38:40.620627+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:08.583512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.610091+0000","last_change":"2024-09-10T22:38:40.607316+0000","last_active":"2024-09-10T22:38:40.610091+0000","last_peered":"2024-09-10T22:38:40.610091+0000","last_clean":"2024-09-10T22:38:40.610091+0000","last_became_active":"2024-09-10T22:38:40.606989+0000","last_became_peered":"2024-09-10T22:38:40.606989+0000","last_unstale":"2024-09-10T22:38:40.610091+0000","last_undegraded":"2024-09-10T22:38:40.610091+0000","last_fullsized":"2024-09-10T22:38:40.610091+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:08.970351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616066+0000","last_change":"2024-09-10T22:38:40.608365+0000","last_active":"2024-09-10T22:38:40.616066+0000","last_peered":"2024-09-10T22:38:40.616066+0000","last_clean":"2024-09-10T22:38:40.616066+0000","last_became_active":"2024-09-10T22:38:40.607758+0000","last_became_peered":"2024-09-10T22:38:40.607758+0000","last_unstale":"2024-09-10T22:38:40.616066+0000","last_undegraded":"2024-09-10T22:38:40.616066+0000","last_fullsized":"2024-09-10T22:38:40.616066+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:20.283263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608766+0000","last_change":"2024-09-10T22:38:40.607815+0000","last_active":"2024-09-10T22:38:40.608766+0000","last_peered":"2024-09-10T22:38:40.608766+0000","last_clean":"2024-09-10T22:38:40.608766+0000","last_became_active":"2024-09-10T22:38:40.607549+0000","last_became_peered":"2024-09-10T22:38:40.607549+0000","last_unstale":"2024-09-10T22:38:40.608766+0000","last_undegraded":"2024-09-10T22:38:40.608766+0000","last_fullsized":"2024-09-10T22:38:40.608766+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:52.827052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607983+0000","last_change":"2024-09-10T22:38:40.606116+0000","last_active":"2024-09-10T22:38:40.607983+0000","last_peered":"2024-09-10T22:38:40.607983+0000","last_clean":"2024-09-10T22:38:40.607983+0000","last_became_active":"2024-09-10T22:38:40.605765+0000","last_became_peered":"2024-09-10T22:38:40.605765+0000","last_unstale":"2024-09-10T22:38:40.607983+0000","last_undegraded":"2024-09-10T22:38:40.607983+0000","last_fullsized":"2024-09-10T22:38:40.607983+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:41.024009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607798+0000","last_change":"2024-09-10T22:38:40.603536+0000","last_active":"2024-09-10T22:38:40.607798+0000","last_peered":"2024-09-10T22:38:40.607798+0000","last_clean":"2024-09-10T22:38:40.607798+0000","last_became_active":"2024-09-10T22:38:40.603242+0000","last_became_peered":"2024-09-10T22:38:40.603242+0000","last_unstale":"2024-09-10T22:38:40.607798+0000","last_undegraded":"2024-09-10T22:38:40.607798+0000","last_fullsized":"2024-09-10T22:38:40.607798+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:09.989910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611088+0000","last_change":"2024-09-10T22:38:40.606799+0000","last_active":"2024-09-10T22:38:40.611088+0000","last_peered":"2024-09-10T22:38:40.611088+0000","last_clean":"2024-09-10T22:38:40.611088+0000","last_became_active":"2024-09-10T22:38:40.606553+0000","last_became_peered":"2024-09-10T22:38:40.606553+0000","last_unstale":"2024-09-10T22:38:40.611088+0000","last_undegraded":"2024-09-10T22:38:40.611088+0000","last_fullsized":"2024-09-10T22:38:40.611088+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:53.338610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590940+0000","last_change":"2024-09-10T22:38:40.596080+0000","last_active":"2024-09-10T22:38:41.590940+0000","last_peered":"2024-09-10T22:38:41.590940+0000","last_clean":"2024-09-10T22:38:41.590940+0000","last_became_active":"2024-09-10T22:38:40.595795+0000","last_became_peered":"2024-09-10T22:38:40.595795+0000","last_unstale":"2024-09-10T22:38:41.590940+0000","last_undegraded":"2024-09-10T22:38:41.590940+0000","last_fullsized":"2024-09-10T22:38:41.590940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.090423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591182+0000","last_change":"2024-09-10T22:38:40.594835+0000","last_active":"2024-09-10T22:38:41.591182+0000","last_peered":"2024-09-10T22:38:41.591182+0000","last_clean":"2024-09-10T22:38:41.591182+0000","last_became_active":"2024-09-10T22:38:40.594533+0000","last_became_peered":"2024-09-10T22:38:40.594533+0000","last_unstale":"2024-09-10T22:38:41.591182+0000","last_undegraded":"2024-09-10T22:38:41.591182+0000","last_fullsized":"2024-09-10T22:38:41.591182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:12.084168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607849+0000","last_change":"2024-09-10T22:38:40.604895+0000","last_active":"2024-09-10T22:38:40.607849+0000","last_peered":"2024-09-10T22:38:40.607849+0000","last_clean":"2024-09-10T22:38:40.607849+0000","last_became_active":"2024-09-10T22:38:40.604335+0000","last_became_peered":"2024-09-10T22:38:40.604335+0000","last_unstale":"2024-09-10T22:38:40.607849+0000","last_undegraded":"2024-09-10T22:38:40.607849+0000","last_fullsized":"2024-09-10T22:38:40.607849+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:11.776292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591435+0000","last_change":"2024-09-10T22:38:40.597022+0000","last_active":"2024-09-10T22:38:41.591435+0000","last_peered":"2024-09-10T22:38:41.591435+0000","last_clean":"2024-09-10T22:38:41.591435+0000","last_became_active":"2024-09-10T22:38:40.596693+0000","last_became_peered":"2024-09-10T22:38:40.596693+0000","last_unstale":"2024-09-10T22:38:41.591435+0000","last_undegraded":"2024-09-10T22:38:41.591435+0000","last_fullsized":"2024-09-10T22:38:41.591435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:42.085973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607678+0000","last_change":"2024-09-10T22:38:40.603896+0000","last_active":"2024-09-10T22:38:40.607678+0000","last_peered":"2024-09-10T22:38:40.607678+0000","last_clean":"2024-09-10T22:38:40.607678+0000","last_became_active":"2024-09-10T22:38:40.602970+0000","last_became_peered":"2024-09-10T22:38:40.602970+0000","last_unstale":"2024-09-10T22:38:40.607678+0000","last_undegraded":"2024-09-10T22:38:40.607678+0000","last_fullsized":"2024-09-10T22:38:40.607678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:45.095703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602395+0000","last_change":"2024-09-10T22:38:40.600178+0000","last_active":"2024-09-10T22:38:40.602395+0000","last_peered":"2024-09-10T22:38:40.602395+0000","last_clean":"2024-09-10T22:38:40.602395+0000","last_became_active":"2024-09-10T22:38:40.599906+0000","last_became_peered":"2024-09-10T22:38:40.599906+0000","last_unstale":"2024-09-10T22:38:40.602395+0000","last_undegraded":"2024-09-10T22:38:40.602395+0000","last_fullsized":"2024-09-10T22:38:40.602395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:24.232203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611393+0000","last_change":"2024-09-10T22:38:40.609977+0000","last_active":"2024-09-10T22:38:40.611393+0000","last_peered":"2024-09-10T22:38:40.611393+0000","last_clean":"2024-09-10T22:38:40.611393+0000","last_became_active":"2024-09-10T22:38:40.609446+0000","last_became_peered":"2024-09-10T22:38:40.609446+0000","last_unstale":"2024-09-10T22:38:40.611393+0000","last_undegraded":"2024-09-10T22:38:40.611393+0000","last_fullsized":"2024-09-10T22:38:40.611393+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:44.400517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.625805+0000","last_change":"2024-09-10T22:38:39.605644+0000","last_active":"2024-09-10T22:38:40.625805+0000","last_peered":"2024-09-10T22:38:40.625805+0000","last_clean":"2024-09-10T22:38:40.625805+0000","last_became_active":"2024-09-10T22:38:39.605193+0000","last_became_peered":"2024-09-10T22:38:39.605193+0000","last_unstale":"2024-09-10T22:38:40.625805+0000","last_undegraded":"2024-09-10T22:38:40.625805+0000","last_fullsized":"2024-09-10T22:38:40.625805+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:10.706373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.605313+0000","last_change":"2024-09-10T22:38:40.602604+0000","last_active":"2024-09-10T22:38:40.605313+0000","last_peered":"2024-09-10T22:38:40.605313+0000","last_clean":"2024-09-10T22:38:40.605313+0000","last_became_active":"2024-09-10T22:38:40.602299+0000","last_became_peered":"2024-09-10T22:38:40.602299+0000","last_unstale":"2024-09-10T22:38:40.605313+0000","last_undegraded":"2024-09-10T22:38:40.605313+0000","last_fullsized":"2024-09-10T22:38:40.605313+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:04.646635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591305+0000","last_change":"2024-09-10T22:38:39.594655+0000","last_active":"2024-09-10T22:38:41.591305+0000","last_peered":"2024-09-10T22:38:41.591305+0000","last_clean":"2024-09-10T22:38:41.591305+0000","last_became_active":"2024-09-10T22:38:39.594350+0000","last_became_peered":"2024-09-10T22:38:39.594350+0000","last_unstale":"2024-09-10T22:38:41.591305+0000","last_undegraded":"2024-09-10T22:38:41.591305+0000","last_fullsized":"2024-09-10T22:38:41.591305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:19.280955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591548+0000","last_change":"2024-09-10T22:38:40.606263+0000","last_active":"2024-09-10T22:38:41.591548+0000","last_peered":"2024-09-10T22:38:41.591548+0000","last_clean":"2024-09-10T22:38:41.591548+0000","last_became_active":"2024-09-10T22:38:40.605955+0000","last_became_peered":"2024-09-10T22:38:40.605955+0000","last_unstale":"2024-09-10T22:38:41.591548+0000","last_undegraded":"2024-09-10T22:38:41.591548+0000","last_fullsized":"2024-09-10T22:38:41.591548+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:02.571771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.610981+0000","last_change":"2024-09-10T22:38:39.599700+0000","last_active":"2024-09-10T22:38:40.610981+0000","last_peered":"2024-09-10T22:38:40.610981+0000","last_clean":"2024-09-10T22:38:40.610981+0000","last_became_active":"2024-09-10T22:38:39.599372+0000","last_became_peered":"2024-09-10T22:38:39.599372+0000","last_unstale":"2024-09-10T22:38:40.610981+0000","last_undegraded":"2024-09-10T22:38:40.610981+0000","last_fullsized":"2024-09-10T22:38:40.610981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:32.967428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611041+0000","last_change":"2024-09-10T22:38:40.606245+0000","last_active":"2024-09-10T22:38:40.611041+0000","last_peered":"2024-09-10T22:38:40.611041+0000","last_clean":"2024-09-10T22:38:40.611041+0000","last_became_active":"2024-09-10T22:38:40.605911+0000","last_became_peered":"2024-09-10T22:38:40.605911+0000","last_unstale":"2024-09-10T22:38:40.611041+0000","last_undegraded":"2024-09-10T22:38:40.611041+0000","last_fullsized":"2024-09-10T22:38:40.611041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:37.382726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.618164+0000","last_change":"2024-09-10T22:38:39.602630+0000","last_active":"2024-09-10T22:38:40.618164+0000","last_peered":"2024-09-10T22:38:40.618164+0000","last_clean":"2024-09-10T22:38:40.618164+0000","last_became_active":"2024-09-10T22:38:39.602348+0000","last_became_peered":"2024-09-10T22:38:39.602348+0000","last_unstale":"2024-09-10T22:38:40.618164+0000","last_undegraded":"2024-09-10T22:38:40.618164+0000","last_fullsized":"2024-09-10T22:38:40.618164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:02.672307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591657+0000","last_change":"2024-09-10T22:38:40.607120+0000","last_active":"2024-09-10T22:38:41.591657+0000","last_peered":"2024-09-10T22:38:41.591657+0000","last_clean":"2024-09-10T22:38:41.591657+0000","last_became_active":"2024-09-10T22:38:40.606811+0000","last_became_peered":"2024-09-10T22:38:40.606811+0000","last_unstale":"2024-09-10T22:38:41.591657+0000","last_undegraded":"2024-09-10T22:38:41.591657+0000","last_fullsized":"2024-09-10T22:38:41.591657+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:39.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.625673+0000","last_change":"2024-09-10T22:38:39.605598+0000","last_active":"2024-09-10T22:38:40.625673+0000","last_peered":"2024-09-10T22:38:40.625673+0000","last_clean":"2024-09-10T22:38:40.625673+0000","last_became_active":"2024-09-10T22:38:39.605276+0000","last_became_peered":"2024-09-10T22:38:39.605276+0000","last_unstale":"2024-09-10T22:38:40.625673+0000","last_undegraded":"2024-09-10T22:38:40.625673+0000","last_fullsized":"2024-09-10T22:38:40.625673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:21.150701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616040+0000","last_change":"2024-09-10T22:38:40.609038+0000","last_active":"2024-09-10T22:38:40.616040+0000","last_peered":"2024-09-10T22:38:40.616040+0000","last_clean":"2024-09-10T22:38:40.616040+0000","last_became_active":"2024-09-10T22:38:40.608759+0000","last_became_peered":"2024-09-10T22:38:40.608759+0000","last_unstale":"2024-09-10T22:38:40.616040+0000","last_undegraded":"2024-09-10T22:38:40.616040+0000","last_fullsized":"2024-09-10T22:38:40.616040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:41.939124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591792+0000","last_change":"2024-09-10T22:38:39.599770+0000","last_active":"2024-09-10T22:38:41.591792+0000","last_peered":"2024-09-10T22:38:41.591792+0000","last_clean":"2024-09-10T22:38:41.591792+0000","last_became_active":"2024-09-10T22:38:39.599481+0000","last_became_peered":"2024-09-10T22:38:39.599481+0000","last_unstale":"2024-09-10T22:38:41.591792+0000","last_undegraded":"2024-09-10T22:38:41.591792+0000","last_fullsized":"2024-09-10T22:38:41.591792+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:40.936816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616555+0000","last_change":"2024-09-10T22:38:40.611882+0000","last_active":"2024-09-10T22:38:40.616555+0000","last_peered":"2024-09-10T22:38:40.616555+0000","last_clean":"2024-09-10T22:38:40.616555+0000","last_became_active":"2024-09-10T22:38:40.608025+0000","last_became_peered":"2024-09-10T22:38:40.608025+0000","last_unstale":"2024-09-10T22:38:40.616555+0000","last_undegraded":"2024-09-10T22:38:40.616555+0000","last_fullsized":"2024-09-10T22:38:40.616555+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:06.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590815+0000","last_change":"2024-09-10T22:38:39.605345+0000","last_active":"2024-09-10T22:38:41.590815+0000","last_peered":"2024-09-10T22:38:41.590815+0000","last_clean":"2024-09-10T22:38:41.590815+0000","last_became_active":"2024-09-10T22:38:39.605027+0000","last_became_peered":"2024-09-10T22:38:39.605027+0000","last_unstale":"2024-09-10T22:38:41.590815+0000","last_undegraded":"2024-09-10T22:38:41.590815+0000","last_fullsized":"2024-09-10T22:38:41.590815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:13.394525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608738+0000","last_change":"2024-09-10T22:38:40.607144+0000","last_active":"2024-09-10T22:38:40.608738+0000","last_peered":"2024-09-10T22:38:40.608738+0000","last_clean":"2024-09-10T22:38:40.608738+0000","last_became_active":"2024-09-10T22:38:40.605350+0000","last_became_peered":"2024-09-10T22:38:40.605350+0000","last_unstale":"2024-09-10T22:38:40.608738+0000","last_undegraded":"2024-09-10T22:38:40.608738+0000","last_fullsized":"2024-09-10T22:38:40.608738+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:07.780094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602377+0000","last_change":"2024-09-10T22:38:39.608819+0000","last_active":"2024-09-10T22:38:40.602377+0000","last_peered":"2024-09-10T22:38:40.602377+0000","last_clean":"2024-09-10T22:38:40.602377+0000","last_became_active":"2024-09-10T22:38:39.608486+0000","last_became_peered":"2024-09-10T22:38:39.608486+0000","last_unstale":"2024-09-10T22:38:40.602377+0000","last_undegraded":"2024-09-10T22:38:40.602377+0000","last_fullsized":"2024-09-10T22:38:40.602377+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:49.482624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591181+0000","last_change":"2024-09-10T22:38:40.612734+0000","last_active":"2024-09-10T22:38:41.591181+0000","last_peered":"2024-09-10T22:38:41.591181+0000","last_clean":"2024-09-10T22:38:41.591181+0000","last_became_active":"2024-09-10T22:38:40.612456+0000","last_became_peered":"2024-09-10T22:38:40.612456+0000","last_unstale":"2024-09-10T22:38:41.591181+0000","last_undegraded":"2024-09-10T22:38:41.591181+0000","last_fullsized":"2024-09-10T22:38:41.591181+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:02.766716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591612+0000","last_change":"2024-09-10T22:38:39.596794+0000","last_active":"2024-09-10T22:38:41.591612+0000","last_peered":"2024-09-10T22:38:41.591612+0000","last_clean":"2024-09-10T22:38:41.591612+0000","last_became_active":"2024-09-10T22:38:39.596516+0000","last_became_peered":"2024-09-10T22:38:39.596516+0000","last_unstale":"2024-09-10T22:38:41.591612+0000","last_undegraded":"2024-09-10T22:38:41.591612+0000","last_fullsized":"2024-09-10T22:38:41.591612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:33.201448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.604972+0000","last_change":"2024-09-10T22:38:40.598896+0000","last_active":"2024-09-10T22:38:40.604972+0000","last_peered":"2024-09-10T22:38:40.604972+0000","last_clean":"2024-09-10T22:38:40.604972+0000","last_became_active":"2024-09-10T22:38:40.598621+0000","last_became_peered":"2024-09-10T22:38:40.598621+0000","last_unstale":"2024-09-10T22:38:40.604972+0000","last_undegraded":"2024-09-10T22:38:40.604972+0000","last_fullsized":"2024-09-10T22:38:40.604972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:13.539196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.603841+0000","last_change":"2024-09-10T22:38:39.609051+0000","last_active":"2024-09-10T22:38:40.603841+0000","last_peered":"2024-09-10T22:38:40.603841+0000","last_clean":"2024-09-10T22:38:40.603841+0000","last_became_active":"2024-09-10T22:38:39.608768+0000","last_became_peered":"2024-09-10T22:38:39.608768+0000","last_unstale":"2024-09-10T22:38:40.603841+0000","last_undegraded":"2024-09-10T22:38:40.603841+0000","last_fullsized":"2024-09-10T22:38:40.603841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:37.387484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616260+0000","last_change":"2024-09-10T22:38:40.610099+0000","last_active":"2024-09-10T22:38:40.616260+0000","last_peered":"2024-09-10T22:38:40.616260+0000","last_clean":"2024-09-10T22:38:40.616260+0000","last_became_active":"2024-09-10T22:38:40.609557+0000","last_became_peered":"2024-09-10T22:38:40.609557+0000","last_unstale":"2024-09-10T22:38:40.616260+0000","last_undegraded":"2024-09-10T22:38:40.616260+0000","last_fullsized":"2024-09-10T22:38:40.616260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:42.245161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.617912+0000","last_change":"2024-09-10T22:38:39.600819+0000","last_active":"2024-09-10T22:38:40.617912+0000","last_peered":"2024-09-10T22:38:40.617912+0000","last_clean":"2024-09-10T22:38:40.617912+0000","last_became_active":"2024-09-10T22:38:39.600215+0000","last_became_peered":"2024-09-10T22:38:39.600215+0000","last_unstale":"2024-09-10T22:38:40.617912+0000","last_undegraded":"2024-09-10T22:38:40.617912+0000","last_fullsized":"2024-09-10T22:38:40.617912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:43.322365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.604757+0000","last_change":"2024-09-10T22:38:40.601356+0000","last_active":"2024-09-10T22:38:40.604757+0000","last_peered":"2024-09-10T22:38:40.604757+0000","last_clean":"2024-09-10T22:38:40.604757+0000","last_became_active":"2024-09-10T22:38:40.601073+0000","last_became_peered":"2024-09-10T22:38:40.601073+0000","last_unstale":"2024-09-10T22:38:40.604757+0000","last_undegraded":"2024-09-10T22:38:40.604757+0000","last_fullsized":"2024-09-10T22:38:40.604757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:46.906395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.627138+0000","last_change":"2024-09-10T22:38:39.593820+0000","last_active":"2024-09-10T22:38:40.627138+0000","last_peered":"2024-09-10T22:38:40.627138+0000","last_clean":"2024-09-10T22:38:40.627138+0000","last_became_active":"2024-09-10T22:38:39.593642+0000","last_became_peered":"2024-09-10T22:38:39.593642+0000","last_unstale":"2024-09-10T22:38:40.627138+0000","last_undegraded":"2024-09-10T22:38:40.627138+0000","last_fullsized":"2024-09-10T22:38:40.627138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.216783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.617782+0000","last_change":"2024-09-10T22:38:40.616137+0000","last_active":"2024-09-10T22:38:40.617782+0000","last_peered":"2024-09-10T22:38:40.617782+0000","last_clean":"2024-09-10T22:38:40.617782+0000","last_became_active":"2024-09-10T22:38:40.615880+0000","last_became_peered":"2024-09-10T22:38:40.615880+0000","last_unstale":"2024-09-10T22:38:40.617782+0000","last_undegraded":"2024-09-10T22:38:40.617782+0000","last_fullsized":"2024-09-10T22:38:40.617782+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:02.924596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.620950+0000","last_change":"2024-09-10T22:38:39.609121+0000","last_active":"2024-09-10T22:38:40.620950+0000","last_peered":"2024-09-10T22:38:40.620950+0000","last_clean":"2024-09-10T22:38:40.620950+0000","last_became_active":"2024-09-10T22:38:39.608803+0000","last_became_peered":"2024-09-10T22:38:39.608803+0000","last_unstale":"2024-09-10T22:38:40.620950+0000","last_undegraded":"2024-09-10T22:38:40.620950+0000","last_fullsized":"2024-09-10T22:38:40.620950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:34.687683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616014+0000","last_change":"2024-09-10T22:38:40.607818+0000","last_active":"2024-09-10T22:38:40.616014+0000","last_peered":"2024-09-10T22:38:40.616014+0000","last_clean":"2024-09-10T22:38:40.616014+0000","last_became_active":"2024-09-10T22:38:40.605884+0000","last_became_peered":"2024-09-10T22:38:40.605884+0000","last_unstale":"2024-09-10T22:38:40.616014+0000","last_undegraded":"2024-09-10T22:38:40.616014+0000","last_fullsized":"2024-09-10T22:38:40.616014+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:34.203756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593296+0000","last_change":"2024-09-10T22:38:39.599483+0000","last_active":"2024-09-10T22:38:40.593296+0000","last_peered":"2024-09-10T22:38:40.593296+0000","last_clean":"2024-09-10T22:38:40.593296+0000","last_became_active":"2024-09-10T22:38:39.599229+0000","last_became_peered":"2024-09-10T22:38:39.599229+0000","last_unstale":"2024-09-10T22:38:40.593296+0000","last_undegraded":"2024-09-10T22:38:40.593296+0000","last_fullsized":"2024-09-10T22:38:40.593296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:51.704795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607582+0000","last_change":"2024-09-10T22:38:40.605081+0000","last_active":"2024-09-10T22:38:40.607582+0000","last_peered":"2024-09-10T22:38:40.607582+0000","last_clean":"2024-09-10T22:38:40.607582+0000","last_became_active":"2024-09-10T22:38:40.604772+0000","last_became_peered":"2024-09-10T22:38:40.604772+0000","last_unstale":"2024-09-10T22:38:40.607582+0000","last_undegraded":"2024-09-10T22:38:40.607582+0000","last_fullsized":"2024-09-10T22:38:40.607582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:00.803195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.596085+0000","last_change":"2024-09-10T22:38:39.609549+0000","last_active":"2024-09-10T22:38:40.596085+0000","last_peered":"2024-09-10T22:38:40.596085+0000","last_clean":"2024-09-10T22:38:40.596085+0000","last_became_active":"2024-09-10T22:38:39.609065+0000","last_became_peered":"2024-09-10T22:38:39.609065+0000","last_unstale":"2024-09-10T22:38:40.596085+0000","last_undegraded":"2024-09-10T22:38:40.596085+0000","last_fullsized":"2024-09-10T22:38:40.596085+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:15.236028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.609911+0000","last_change":"2024-09-10T22:38:40.606321+0000","last_active":"2024-09-10T22:38:40.609911+0000","last_peered":"2024-09-10T22:38:40.609911+0000","last_clean":"2024-09-10T22:38:40.609911+0000","last_became_active":"2024-09-10T22:38:40.605852+0000","last_became_peered":"2024-09-10T22:38:40.605852+0000","last_unstale":"2024-09-10T22:38:40.609911+0000","last_undegraded":"2024-09-10T22:38:40.609911+0000","last_fullsized":"2024-09-10T22:38:40.609911+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:39.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.623199+0000","last_change":"2024-09-10T22:38:39.608173+0000","last_active":"2024-09-10T22:38:40.623199+0000","last_peered":"2024-09-10T22:38:40.623199+0000","last_clean":"2024-09-10T22:38:40.623199+0000","last_became_active":"2024-09-10T22:38:39.607830+0000","last_became_peered":"2024-09-10T22:38:39.607830+0000","last_unstale":"2024-09-10T22:38:40.623199+0000","last_undegraded":"2024-09-10T22:38:40.623199+0000","last_fullsized":"2024-09-10T22:38:40.623199+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:39.333312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.601704+0000","last_change":"2024-09-10T22:38:40.593482+0000","last_active":"2024-09-10T22:38:40.601704+0000","last_peered":"2024-09-10T22:38:40.601704+0000","last_clean":"2024-09-10T22:38:40.601704+0000","last_became_active":"2024-09-10T22:38:40.593215+0000","last_became_peered":"2024-09-10T22:38:40.593215+0000","last_unstale":"2024-09-10T22:38:40.601704+0000","last_undegraded":"2024-09-10T22:38:40.601704+0000","last_fullsized":"2024-09-10T22:38:40.601704+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:29.594498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.626090+0000","last_change":"2024-09-10T22:38:39.607157+0000","last_active":"2024-09-10T22:38:40.626090+0000","last_peered":"2024-09-10T22:38:40.626090+0000","last_clean":"2024-09-10T22:38:40.626090+0000","last_became_active":"2024-09-10T22:38:39.606835+0000","last_became_peered":"2024-09-10T22:38:39.606835+0000","last_unstale":"2024-09-10T22:38:40.626090+0000","last_undegraded":"2024-09-10T22:38:40.626090+0000","last_fullsized":"2024-09-10T22:38:40.626090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:05.761073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.618297+0000","last_change":"2024-09-10T22:38:40.612589+0000","last_active":"2024-09-10T22:38:40.618297+0000","last_peered":"2024-09-10T22:38:40.618297+0000","last_clean":"2024-09-10T22:38:40.618297+0000","last_became_active":"2024-09-10T22:38:40.612311+0000","last_became_peered":"2024-09-10T22:38:40.612311+0000","last_unstale":"2024-09-10T22:38:40.618297+0000","last_undegraded":"2024-09-10T22:38:40.618297+0000","last_fullsized":"2024-09-10T22:38:40.618297+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:59.412309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591422+0000","last_change":"2024-09-10T22:38:39.604556+0000","last_active":"2024-09-10T22:38:41.591422+0000","last_peered":"2024-09-10T22:38:41.591422+0000","last_clean":"2024-09-10T22:38:41.591422+0000","last_became_active":"2024-09-10T22:38:39.604119+0000","last_became_peered":"2024-09-10T22:38:39.604119+0000","last_unstale":"2024-09-10T22:38:41.591422+0000","last_undegraded":"2024-09-10T22:38:41.591422+0000","last_fullsized":"2024-09-10T22:38:41.591422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:02.516476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611256+0000","last_change":"2024-09-10T22:38:40.609572+0000","last_active":"2024-09-10T22:38:40.611256+0000","last_peered":"2024-09-10T22:38:40.611256+0000","last_clean":"2024-09-10T22:38:40.611256+0000","last_became_active":"2024-09-10T22:38:40.608929+0000","last_became_peered":"2024-09-10T22:38:40.608929+0000","last_unstale":"2024-09-10T22:38:40.611256+0000","last_undegraded":"2024-09-10T22:38:40.611256+0000","last_fullsized":"2024-09-10T22:38:40.611256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:54.564665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591125+0000","last_change":"2024-09-10T22:38:39.605022+0000","last_active":"2024-09-10T22:38:41.591125+0000","last_peered":"2024-09-10T22:38:41.591125+0000","last_clean":"2024-09-10T22:38:41.591125+0000","last_became_active":"2024-09-10T22:38:39.604688+0000","last_became_peered":"2024-09-10T22:38:39.604688+0000","last_unstale":"2024-09-10T22:38:41.591125+0000","last_undegraded":"2024-09-10T22:38:41.591125+0000","last_fullsized":"2024-09-10T22:38:41.591125+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:55.144203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607789+0000","last_change":"2024-09-10T22:38:40.604783+0000","last_active":"2024-09-10T22:38:40.607789+0000","last_peered":"2024-09-10T22:38:40.607789+0000","last_clean":"2024-09-10T22:38:40.607789+0000","last_became_active":"2024-09-10T22:38:40.604447+0000","last_became_peered":"2024-09-10T22:38:40.604447+0000","last_unstale":"2024-09-10T22:38:40.607789+0000","last_undegraded":"2024-09-10T22:38:40.607789+0000","last_fullsized":"2024-09-10T22:38:40.607789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:20.249674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.596706+0000","last_change":"2024-09-10T22:38:39.606706+0000","last_active":"2024-09-10T22:38:40.596706+0000","last_peered":"2024-09-10T22:38:40.596706+0000","last_clean":"2024-09-10T22:38:40.596706+0000","last_became_active":"2024-09-10T22:38:39.606414+0000","last_became_peered":"2024-09-10T22:38:39.606414+0000","last_unstale":"2024-09-10T22:38:40.596706+0000","last_undegraded":"2024-09-10T22:38:40.596706+0000","last_fullsized":"2024-09-10T22:38:40.596706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:15.835251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.614056+0000","last_change":"2024-09-10T22:38:40.610133+0000","last_active":"2024-09-10T22:38:40.614056+0000","last_peered":"2024-09-10T22:38:40.614056+0000","last_clean":"2024-09-10T22:38:40.614056+0000","last_became_active":"2024-09-10T22:38:40.609526+0000","last_became_peered":"2024-09-10T22:38:40.609526+0000","last_unstale":"2024-09-10T22:38:40.614056+0000","last_undegraded":"2024-09-10T22:38:40.614056+0000","last_fullsized":"2024-09-10T22:38:40.614056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:08.333001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.625759+0000","last_change":"2024-09-10T22:38:39.603931+0000","last_active":"2024-09-10T22:38:40.625759+0000","last_peered":"2024-09-10T22:38:40.625759+0000","last_clean":"2024-09-10T22:38:40.625759+0000","last_became_active":"2024-09-10T22:38:39.603658+0000","last_became_peered":"2024-09-10T22:38:39.603658+0000","last_unstale":"2024-09-10T22:38:40.625759+0000","last_undegraded":"2024-09-10T22:38:40.625759+0000","last_fullsized":"2024-09-10T22:38:40.625759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:41.825217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.609856+0000","last_change":"2024-09-10T22:38:40.605841+0000","last_active":"2024-09-10T22:38:40.609856+0000","last_peered":"2024-09-10T22:38:40.609856+0000","last_clean":"2024-09-10T22:38:40.609856+0000","last_became_active":"2024-09-10T22:38:40.605467+0000","last_became_peered":"2024-09-10T22:38:40.605467+0000","last_unstale":"2024-09-10T22:38:40.609856+0000","last_undegraded":"2024-09-10T22:38:40.609856+0000","last_fullsized":"2024-09-10T22:38:40.609856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:40.816546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590789+0000","last_change":"2024-09-10T22:38:39.598825+0000","last_active":"2024-09-10T22:38:41.590789+0000","last_peered":"2024-09-10T22:38:41.590789+0000","last_clean":"2024-09-10T22:38:41.590789+0000","last_became_active":"2024-09-10T22:38:39.598473+0000","last_became_peered":"2024-09-10T22:38:39.598473+0000","last_unstale":"2024-09-10T22:38:41.590789+0000","last_undegraded":"2024-09-10T22:38:41.590789+0000","last_fullsized":"2024-09-10T22:38:41.590789+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:04.427415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602370+0000","last_change":"2024-09-10T22:38:40.599523+0000","last_active":"2024-09-10T22:38:40.602370+0000","last_peered":"2024-09-10T22:38:40.602370+0000","last_clean":"2024-09-10T22:38:40.602370+0000","last_became_active":"2024-09-10T22:38:40.599241+0000","last_became_peered":"2024-09-10T22:38:40.599241+0000","last_unstale":"2024-09-10T22:38:40.602370+0000","last_undegraded":"2024-09-10T22:38:40.602370+0000","last_fullsized":"2024-09-10T22:38:40.602370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:14.083091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.599230+0000","last_change":"2024-09-10T22:38:39.608091+0000","last_active":"2024-09-10T22:38:40.599230+0000","last_peered":"2024-09-10T22:38:40.599230+0000","last_clean":"2024-09-10T22:38:40.599230+0000","last_became_active":"2024-09-10T22:38:39.607826+0000","last_became_peered":"2024-09-10T22:38:39.607826+0000","last_unstale":"2024-09-10T22:38:40.599230+0000","last_undegraded":"2024-09-10T22:38:40.599230+0000","last_fullsized":"2024-09-10T22:38:40.599230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:51.824744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.620459+0000","last_change":"2024-09-10T22:38:40.614672+0000","last_active":"2024-09-10T22:38:40.620459+0000","last_peered":"2024-09-10T22:38:40.620459+0000","last_clean":"2024-09-10T22:38:40.620459+0000","last_became_active":"2024-09-10T22:38:40.614357+0000","last_became_peered":"2024-09-10T22:38:40.614357+0000","last_unstale":"2024-09-10T22:38:40.620459+0000","last_undegraded":"2024-09-10T22:38:40.620459+0000","last_fullsized":"2024-09-10T22:38:40.620459+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:52.707103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593649+0000","last_change":"2024-09-10T22:38:39.602385+0000","last_active":"2024-09-10T22:38:40.593649+0000","last_peered":"2024-09-10T22:38:40.593649+0000","last_clean":"2024-09-10T22:38:40.593649+0000","last_became_active":"2024-09-10T22:38:39.601901+0000","last_became_peered":"2024-09-10T22:38:39.601901+0000","last_unstale":"2024-09-10T22:38:40.593649+0000","last_undegraded":"2024-09-10T22:38:40.593649+0000","last_fullsized":"2024-09-10T22:38:40.593649+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:48.909028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.613680+0000","last_change":"2024-09-10T22:38:40.612463+0000","last_active":"2024-09-10T22:38:40.613680+0000","last_peered":"2024-09-10T22:38:40.613680+0000","last_clean":"2024-09-10T22:38:40.613680+0000","last_became_active":"2024-09-10T22:38:40.612148+0000","last_became_peered":"2024-09-10T22:38:40.612148+0000","last_unstale":"2024-09-10T22:38:40.613680+0000","last_undegraded":"2024-09-10T22:38:40.613680+0000","last_fullsized":"2024-09-10T22:38:40.613680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:50.484932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.595666+0000","last_change":"2024-09-10T22:38:39.605995+0000","last_active":"2024-09-10T22:38:40.595666+0000","last_peered":"2024-09-10T22:38:40.595666+0000","last_clean":"2024-09-10T22:38:40.595666+0000","last_became_active":"2024-09-10T22:38:39.604806+0000","last_became_peered":"2024-09-10T22:38:39.604806+0000","last_unstale":"2024-09-10T22:38:40.595666+0000","last_undegraded":"2024-09-10T22:38:40.595666+0000","last_fullsized":"2024-09-10T22:38:40.595666+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:15.502820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608912+0000","last_change":"2024-09-10T22:38:40.602467+0000","last_active":"2024-09-10T22:38:40.608912+0000","last_peered":"2024-09-10T22:38:40.608912+0000","last_clean":"2024-09-10T22:38:40.608912+0000","last_became_active":"2024-09-10T22:38:40.602131+0000","last_became_peered":"2024-09-10T22:38:40.602131+0000","last_unstale":"2024-09-10T22:38:40.608912+0000","last_undegraded":"2024-09-10T22:38:40.608912+0000","last_fullsized":"2024-09-10T22:38:40.608912+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:52.302762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.594567+0000","last_change":"2024-09-10T22:38:39.600489+0000","last_active":"2024-09-10T22:38:40.594567+0000","last_peered":"2024-09-10T22:38:40.594567+0000","last_clean":"2024-09-10T22:38:40.594567+0000","last_became_active":"2024-09-10T22:38:39.599888+0000","last_became_peered":"2024-09-10T22:38:39.599888+0000","last_unstale":"2024-09-10T22:38:40.594567+0000","last_undegraded":"2024-09-10T22:38:40.594567+0000","last_fullsized":"2024-09-10T22:38:40.594567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.450705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.605550+0000","last_change":"2024-09-10T22:38:40.603701+0000","last_active":"2024-09-10T22:38:40.605550+0000","last_peered":"2024-09-10T22:38:40.605550+0000","last_clean":"2024-09-10T22:38:40.605550+0000","last_became_active":"2024-09-10T22:38:40.603445+0000","last_became_peered":"2024-09-10T22:38:40.603445+0000","last_unstale":"2024-09-10T22:38:40.605550+0000","last_undegraded":"2024-09-10T22:38:40.605550+0000","last_fullsized":"2024-09-10T22:38:40.605550+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:42.085973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.597906+0000","last_change":"2024-09-10T22:38:39.603987+0000","last_active":"2024-09-10T22:38:40.597906+0000","last_peered":"2024-09-10T22:38:40.597906+0000","last_clean":"2024-09-10T22:38:40.597906+0000","last_became_active":"2024-09-10T22:38:39.603630+0000","last_became_peered":"2024-09-10T22:38:39.603630+0000","last_unstale":"2024-09-10T22:38:40.597906+0000","last_undegraded":"2024-09-10T22:38:40.597906+0000","last_fullsized":"2024-09-10T22:38:40.597906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:11.073421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607327+0000","last_change":"2024-09-10T22:38:40.602737+0000","last_active":"2024-09-10T22:38:40.607327+0000","last_peered":"2024-09-10T22:38:40.607327+0000","last_clean":"2024-09-10T22:38:40.607327+0000","last_became_active":"2024-09-10T22:38:40.602396+0000","last_became_peered":"2024-09-10T22:38:40.602396+0000","last_unstale":"2024-09-10T22:38:40.607327+0000","last_undegraded":"2024-09-10T22:38:40.607327+0000","last_fullsized":"2024-09-10T22:38:40.607327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:02.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593568+0000","last_change":"2024-09-10T22:38:39.597398+0000","last_active":"2024-09-10T22:38:40.593568+0000","last_peered":"2024-09-10T22:38:40.593568+0000","last_clean":"2024-09-10T22:38:40.593568+0000","last_became_active":"2024-09-10T22:38:39.597124+0000","last_became_peered":"2024-09-10T22:38:39.597124+0000","last_unstale":"2024-09-10T22:38:40.593568+0000","last_undegraded":"2024-09-10T22:38:40.593568+0000","last_fullsized":"2024-09-10T22:38:40.593568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:14.150710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607913+0000","last_change":"2024-09-10T22:38:40.605128+0000","last_active":"2024-09-10T22:38:40.607913+0000","last_peered":"2024-09-10T22:38:40.607913+0000","last_clean":"2024-09-10T22:38:40.607913+0000","last_became_active":"2024-09-10T22:38:40.604859+0000","last_became_peered":"2024-09-10T22:38:40.604859+0000","last_unstale":"2024-09-10T22:38:40.607913+0000","last_undegraded":"2024-09-10T22:38:40.607913+0000","last_fullsized":"2024-09-10T22:38:40.607913+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:21.112573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591895+0000","last_change":"2024-09-10T22:38:39.606428+0000","last_active":"2024-09-10T22:38:41.591895+0000","last_peered":"2024-09-10T22:38:41.591895+0000","last_clean":"2024-09-10T22:38:41.591895+0000","last_became_active":"2024-09-10T22:38:39.606030+0000","last_became_peered":"2024-09-10T22:38:39.606030+0000","last_unstale":"2024-09-10T22:38:41.591895+0000","last_undegraded":"2024-09-10T22:38:41.591895+0000","last_fullsized":"2024-09-10T22:38:41.591895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:05.772523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590641+0000","last_change":"2024-09-10T22:38:40.600176+0000","last_active":"2024-09-10T22:38:41.590641+0000","last_peered":"2024-09-10T22:38:41.590641+0000","last_clean":"2024-09-10T22:38:41.590641+0000","last_became_active":"2024-09-10T22:38:40.599888+0000","last_became_peered":"2024-09-10T22:38:40.599888+0000","last_unstale":"2024-09-10T22:38:41.590641+0000","last_undegraded":"2024-09-10T22:38:41.590641+0000","last_fullsized":"2024-09-10T22:38:41.590641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:05.125719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.597881+0000","last_change":"2024-09-10T22:38:39.601705+0000","last_active":"2024-09-10T22:38:40.597881+0000","last_peered":"2024-09-10T22:38:40.597881+0000","last_clean":"2024-09-10T22:38:40.597881+0000","last_became_active":"2024-09-10T22:38:39.601319+0000","last_became_peered":"2024-09-10T22:38:39.601319+0000","last_unstale":"2024-09-10T22:38:40.597881+0000","last_undegraded":"2024-09-10T22:38:40.597881+0000","last_fullsized":"2024-09-10T22:38:40.597881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:04.818960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590640+0000","last_change":"2024-09-10T22:38:40.602352+0000","last_active":"2024-09-10T22:38:41.590640+0000","last_peered":"2024-09-10T22:38:41.590640+0000","last_clean":"2024-09-10T22:38:41.590640+0000","last_became_active":"2024-09-10T22:38:40.601968+0000","last_became_peered":"2024-09-10T22:38:40.601968+0000","last_unstale":"2024-09-10T22:38:41.590640+0000","last_undegraded":"2024-09-10T22:38:41.590640+0000","last_fullsized":"2024-09-10T22:38:41.590640+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:44.324673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602827+0000","last_change":"2024-09-10T22:38:39.611242+0000","last_active":"2024-09-10T22:38:40.602827+0000","last_peered":"2024-09-10T22:38:40.602827+0000","last_clean":"2024-09-10T22:38:40.602827+0000","last_became_active":"2024-09-10T22:38:39.610940+0000","last_became_peered":"2024-09-10T22:38:39.610940+0000","last_unstale":"2024-09-10T22:38:40.602827+0000","last_undegraded":"2024-09-10T22:38:40.602827+0000","last_fullsized":"2024-09-10T22:38:40.602827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:01.922287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608082+0000","last_change":"2024-09-10T22:38:40.601507+0000","last_active":"2024-09-10T22:38:40.608082+0000","last_peered":"2024-09-10T22:38:40.608082+0000","last_clean":"2024-09-10T22:38:40.608082+0000","last_became_active":"2024-09-10T22:38:40.601224+0000","last_became_peered":"2024-09-10T22:38:40.601224+0000","last_unstale":"2024-09-10T22:38:40.608082+0000","last_undegraded":"2024-09-10T22:38:40.608082+0000","last_fullsized":"2024-09-10T22:38:40.608082+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:54.564665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590921+0000","last_change":"2024-09-10T22:38:39.606569+0000","last_active":"2024-09-10T22:38:41.590921+0000","last_peered":"2024-09-10T22:38:41.590921+0000","last_clean":"2024-09-10T22:38:41.590921+0000","last_became_active":"2024-09-10T22:38:39.606253+0000","last_became_peered":"2024-09-10T22:38:39.606253+0000","last_unstale":"2024-09-10T22:38:41.590921+0000","last_undegraded":"2024-09-10T22:38:41.590921+0000","last_fullsized":"2024-09-10T22:38:41.590921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:03.427787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593720+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:38:40.593720+0000","last_peered":"2024-09-10T22:38:40.593720+0000","last_clean":"2024-09-10T22:38:40.593720+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:38:40.593720+0000","last_undegraded":"2024-09-10T22:38:40.593720+0000","last_fullsized":"2024-09-10T22:38:40.593720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1130496,"data_stored":807171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":217385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":216783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":218325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1130496,"data_stored":807115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":520192,"data_stored":209938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":209938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":520192,"data_stored":209938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:38:44.629 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:38:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.769+0000 7ff5e2047640 1 -- 172.21.15.66:0/40609230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 msgr2=0x7ff5c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.769+0000 7ff5e2047640 1 --2- 172.21.15.66:0/40609230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5c400b9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4001dc0 tx=0x7ff5c4005830 comp rx=0 tx=0).stop 2024-09-10T22:38:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.769+0000 7ff5e2047640 1 -- 172.21.15.66:0/40609230 shutdown_connections 2024-09-10T22:38:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.769+0000 7ff5e2047640 1 --2- 172.21.15.66:0/40609230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5c400b9c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.769+0000 7ff5e2047640 1 -- 172.21.15.66:0/40609230 >> 172.21.15.66:0/40609230 conn(0x7ff5dc0d4000 msgr2=0x7ff5dc0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.769+0000 7ff5e2047640 1 -- 172.21.15.66:0/40609230 shutdown_connections 2024-09-10T22:38:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.769+0000 7ff5e2047640 1 -- 172.21.15.66:0/40609230 wait complete. 2024-09-10T22:38:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.770+0000 7ff5e2047640 1 Processor -- start 2024-09-10T22:38:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.771+0000 7ff5e2047640 1 -- start start 2024-09-10T22:38:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.771+0000 7ff5e2047640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5dc0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.771+0000 7ff5e2047640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1472f0 0x7ff5dc0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.771+0000 7ff5e2047640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5dc0cdbe0 0x7ff5dc0b8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.771+0000 7ff5e2047640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc0d0840 con 0x7ff5c40053c0 2024-09-10T22:38:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.771+0000 7ff5e2047640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc0d4e40 con 0x7ff5dc1472f0 2024-09-10T22:38:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.771+0000 7ff5e2047640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff5dc0d09c0 con 0x7ff5dc0cdbe0 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5dc0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5dc0cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48590/0 (socket says 172.21.15.66:48590) 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 -- 172.21.15.66:0/1605819870 learned_addr learned my addr 172.21.15.66:0/1605819870 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5daffd640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1472f0 0x7ff5dc0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5dbfff640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5dc0cdbe0 0x7ff5dc0b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 -- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5dc0cdbe0 msgr2=0x7ff5dc0b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5dc0cdbe0 0x7ff5dc0b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 -- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1472f0 msgr2=0x7ff5dc0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1472f0 0x7ff5dc0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 -- 172.21.15.66:0/1605819870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc0cced0 con 0x7ff5c40053c0 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5dbfff640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5dc0cdbe0 0x7ff5dc0b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5daffd640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1472f0 0x7ff5dc0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5db7fe640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5dc0cd160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc011ae0 tx=0x7ff5cc009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5c3fff640 1 -- 172.21.15.66:0/1605819870 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc01e070 con 0x7ff5c40053c0 2024-09-10T22:38:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5dc0b6ed0 con 0x7ff5c40053c0 2024-09-10T22:38:44.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5c3fff640 1 -- 172.21.15.66:0/1605819870 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5cc004480 con 0x7ff5c40053c0 2024-09-10T22:38:44.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5c3fff640 1 -- 172.21.15.66:0/1605819870 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00fc60 con 0x7ff5c40053c0 2024-09-10T22:38:44.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.772+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5dc15ec60 con 0x7ff5c40053c0 2024-09-10T22:38:44.776 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.773+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5dc0b73f0 con 0x7ff5c40053c0 2024-09-10T22:38:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.774+0000 7ff5c3fff640 1 -- 172.21.15.66:0/1605819870 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff5cc0035f0 con 0x7ff5c40053c0 2024-09-10T22:38:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.774+0000 7ff5c3fff640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.774+0000 7ff5c3fff640 1 -- 172.21.15.66:0/1605819870 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7ff5cc09ab00 con 0x7ff5c40053c0 2024-09-10T22:38:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.775+0000 7ff5daffd640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.775+0000 7ff5daffd640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0a2110 tx=0x7ff5d00068d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:44.779 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.779+0000 7ff5c3fff640 1 -- 172.21.15.66:0/1605819870 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff5dc0b73f0 con 0x7ff5c40053c0 2024-09-10T22:38:44.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.964+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5dc15d5f0 con 0x7ff5c8077fa0 2024-09-10T22:38:44.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.969+0000 7ff5c3fff640 1 -- 172.21.15.66:0/1605819870 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7ff5dc15d5f0 con 0x7ff5c8077fa0 2024-09-10T22:38:44.973 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:44.974 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:38:44.975 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.972+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 msgr2=0x7ff5c807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.972+0000 7ff5e2047640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0a2110 tx=0x7ff5d00068d0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.972+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 msgr2=0x7ff5dc0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.972+0000 7ff5e2047640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5dc0cd160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc011ae0 tx=0x7ff5cc009e40 comp rx=0 tx=0).stop 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 shutdown_connections 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff5c8077fa0 0x7ff5c807a460 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff5dc0cdbe0 0x7ff5dc0b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1472f0 0x7ff5dc0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 --2- 172.21.15.66:0/1605819870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5c40053c0 0x7ff5dc0cd160 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 >> 172.21.15.66:0/1605819870 conn(0x7ff5dc0d4000 msgr2=0x7ff5dc0b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 shutdown_connections 2024-09-10T22:38:44.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:44.973+0000 7ff5e2047640 1 -- 172.21.15.66:0/1605819870 wait complete. 2024-09-10T22:38:44.985 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-10T22:38:44.648919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242396,"kb_used_data":5316,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732324,"statfs":{"total":773094113280,"available":772845899776,"internally_reserved":0,"allocated":5443584,"data_stored":2896593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001466"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.603728+0000","last_change":"2024-09-10T22:38:40.597900+0000","last_active":"2024-09-10T22:38:40.603728+0000","last_peered":"2024-09-10T22:38:40.603728+0000","last_clean":"2024-09-10T22:38:40.603728+0000","last_became_active":"2024-09-10T22:38:40.597608+0000","last_became_peered":"2024-09-10T22:38:40.597608+0000","last_unstale":"2024-09-10T22:38:40.603728+0000","last_undegraded":"2024-09-10T22:38:40.603728+0000","last_fullsized":"2024-09-10T22:38:40.603728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.090423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590792+0000","last_change":"2024-09-10T22:38:40.610451+0000","last_active":"2024-09-10T22:38:41.590792+0000","last_peered":"2024-09-10T22:38:41.590792+0000","last_clean":"2024-09-10T22:38:41.590792+0000","last_became_active":"2024-09-10T22:38:40.610162+0000","last_became_peered":"2024-09-10T22:38:40.610162+0000","last_unstale":"2024-09-10T22:38:41.590792+0000","last_undegraded":"2024-09-10T22:38:41.590792+0000","last_fullsized":"2024-09-10T22:38:41.590792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:12.308156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590708+0000","last_change":"2024-09-10T22:38:40.600360+0000","last_active":"2024-09-10T22:38:41.590708+0000","last_peered":"2024-09-10T22:38:41.590708+0000","last_clean":"2024-09-10T22:38:41.590708+0000","last_became_active":"2024-09-10T22:38:40.600094+0000","last_became_peered":"2024-09-10T22:38:40.600094+0000","last_unstale":"2024-09-10T22:38:41.590708+0000","last_undegraded":"2024-09-10T22:38:41.590708+0000","last_fullsized":"2024-09-10T22:38:41.590708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:48.453013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.615684+0000","last_change":"2024-09-10T22:38:40.612141+0000","last_active":"2024-09-10T22:38:40.615684+0000","last_peered":"2024-09-10T22:38:40.615684+0000","last_clean":"2024-09-10T22:38:40.615684+0000","last_became_active":"2024-09-10T22:38:40.609061+0000","last_became_peered":"2024-09-10T22:38:40.609061+0000","last_unstale":"2024-09-10T22:38:40.615684+0000","last_undegraded":"2024-09-10T22:38:40.615684+0000","last_fullsized":"2024-09-10T22:38:40.615684+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:11.983496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590703+0000","last_change":"2024-09-10T22:38:40.602735+0000","last_active":"2024-09-10T22:38:41.590703+0000","last_peered":"2024-09-10T22:38:41.590703+0000","last_clean":"2024-09-10T22:38:41.590703+0000","last_became_active":"2024-09-10T22:38:40.602387+0000","last_became_peered":"2024-09-10T22:38:40.602387+0000","last_unstale":"2024-09-10T22:38:41.590703+0000","last_undegraded":"2024-09-10T22:38:41.590703+0000","last_fullsized":"2024-09-10T22:38:41.590703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:14.101065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607645+0000","last_change":"2024-09-10T22:38:40.606772+0000","last_active":"2024-09-10T22:38:40.607645+0000","last_peered":"2024-09-10T22:38:40.607645+0000","last_clean":"2024-09-10T22:38:40.607645+0000","last_became_active":"2024-09-10T22:38:40.606525+0000","last_became_peered":"2024-09-10T22:38:40.606525+0000","last_unstale":"2024-09-10T22:38:40.607645+0000","last_undegraded":"2024-09-10T22:38:40.607645+0000","last_fullsized":"2024-09-10T22:38:40.607645+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:21.504845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.604879+0000","last_change":"2024-09-10T22:38:40.601289+0000","last_active":"2024-09-10T22:38:40.604879+0000","last_peered":"2024-09-10T22:38:40.604879+0000","last_clean":"2024-09-10T22:38:40.604879+0000","last_became_active":"2024-09-10T22:38:40.601024+0000","last_became_peered":"2024-09-10T22:38:40.601024+0000","last_unstale":"2024-09-10T22:38:40.604879+0000","last_undegraded":"2024-09-10T22:38:40.604879+0000","last_fullsized":"2024-09-10T22:38:40.604879+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:02.766716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616580+0000","last_change":"2024-09-10T22:38:40.612094+0000","last_active":"2024-09-10T22:38:40.616580+0000","last_peered":"2024-09-10T22:38:40.616580+0000","last_clean":"2024-09-10T22:38:40.616580+0000","last_became_active":"2024-09-10T22:38:40.609155+0000","last_became_peered":"2024-09-10T22:38:40.609155+0000","last_unstale":"2024-09-10T22:38:40.616580+0000","last_undegraded":"2024-09-10T22:38:40.616580+0000","last_fullsized":"2024-09-10T22:38:40.616580+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:09.802304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.623209+0000","last_change":"2024-09-10T22:38:40.618587+0000","last_active":"2024-09-10T22:38:40.623209+0000","last_peered":"2024-09-10T22:38:40.623209+0000","last_clean":"2024-09-10T22:38:40.623209+0000","last_became_active":"2024-09-10T22:38:40.616718+0000","last_became_peered":"2024-09-10T22:38:40.616718+0000","last_unstale":"2024-09-10T22:38:40.623209+0000","last_undegraded":"2024-09-10T22:38:40.623209+0000","last_fullsized":"2024-09-10T22:38:40.623209+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:03.674615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616584+0000","last_change":"2024-09-10T22:38:40.612791+0000","last_active":"2024-09-10T22:38:40.616584+0000","last_peered":"2024-09-10T22:38:40.616584+0000","last_clean":"2024-09-10T22:38:40.616584+0000","last_became_active":"2024-09-10T22:38:40.612523+0000","last_became_peered":"2024-09-10T22:38:40.612523+0000","last_unstale":"2024-09-10T22:38:40.616584+0000","last_undegraded":"2024-09-10T22:38:40.616584+0000","last_fullsized":"2024-09-10T22:38:40.616584+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.241720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.615725+0000","last_change":"2024-09-10T22:38:40.606158+0000","last_active":"2024-09-10T22:38:40.615725+0000","last_peered":"2024-09-10T22:38:40.615725+0000","last_clean":"2024-09-10T22:38:40.615725+0000","last_became_active":"2024-09-10T22:38:40.605704+0000","last_became_peered":"2024-09-10T22:38:40.605704+0000","last_unstale":"2024-09-10T22:38:40.615725+0000","last_undegraded":"2024-09-10T22:38:40.615725+0000","last_fullsized":"2024-09-10T22:38:40.615725+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:07:11.888448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591653+0000","last_change":"2024-09-10T22:38:40.595559+0000","last_active":"2024-09-10T22:38:41.591653+0000","last_peered":"2024-09-10T22:38:41.591653+0000","last_clean":"2024-09-10T22:38:41.591653+0000","last_became_active":"2024-09-10T22:38:40.595295+0000","last_became_peered":"2024-09-10T22:38:40.595295+0000","last_unstale":"2024-09-10T22:38:41.591653+0000","last_undegraded":"2024-09-10T22:38:41.591653+0000","last_fullsized":"2024-09-10T22:38:41.591653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:57.734414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607841+0000","last_change":"2024-09-10T22:38:40.605923+0000","last_active":"2024-09-10T22:38:40.607841+0000","last_peered":"2024-09-10T22:38:40.607841+0000","last_clean":"2024-09-10T22:38:40.607841+0000","last_became_active":"2024-09-10T22:38:40.605695+0000","last_became_peered":"2024-09-10T22:38:40.605695+0000","last_unstale":"2024-09-10T22:38:40.607841+0000","last_undegraded":"2024-09-10T22:38:40.607841+0000","last_fullsized":"2024-09-10T22:38:40.607841+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:08.139735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616091+0000","last_change":"2024-09-10T22:38:40.608665+0000","last_active":"2024-09-10T22:38:40.616091+0000","last_peered":"2024-09-10T22:38:40.616091+0000","last_clean":"2024-09-10T22:38:40.616091+0000","last_became_active":"2024-09-10T22:38:40.608070+0000","last_became_peered":"2024-09-10T22:38:40.608070+0000","last_unstale":"2024-09-10T22:38:40.616091+0000","last_undegraded":"2024-09-10T22:38:40.616091+0000","last_fullsized":"2024-09-10T22:38:40.616091+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:03.518784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607726+0000","last_change":"2024-09-10T22:38:40.604613+0000","last_active":"2024-09-10T22:38:40.607726+0000","last_peered":"2024-09-10T22:38:40.607726+0000","last_clean":"2024-09-10T22:38:40.607726+0000","last_became_active":"2024-09-10T22:38:40.604264+0000","last_became_peered":"2024-09-10T22:38:40.604264+0000","last_unstale":"2024-09-10T22:38:40.607726+0000","last_undegraded":"2024-09-10T22:38:40.607726+0000","last_fullsized":"2024-09-10T22:38:40.607726+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:08.814068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.613699+0000","last_change":"2024-09-10T22:38:40.612699+0000","last_active":"2024-09-10T22:38:40.613699+0000","last_peered":"2024-09-10T22:38:40.613699+0000","last_clean":"2024-09-10T22:38:40.613699+0000","last_became_active":"2024-09-10T22:38:40.612229+0000","last_became_peered":"2024-09-10T22:38:40.612229+0000","last_unstale":"2024-09-10T22:38:40.613699+0000","last_undegraded":"2024-09-10T22:38:40.613699+0000","last_fullsized":"2024-09-10T22:38:40.613699+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:52.302762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591415+0000","last_change":"2024-09-10T22:38:40.594799+0000","last_active":"2024-09-10T22:38:41.591415+0000","last_peered":"2024-09-10T22:38:41.591415+0000","last_clean":"2024-09-10T22:38:41.591415+0000","last_became_active":"2024-09-10T22:38:40.593905+0000","last_became_peered":"2024-09-10T22:38:40.593905+0000","last_unstale":"2024-09-10T22:38:41.591415+0000","last_undegraded":"2024-09-10T22:38:41.591415+0000","last_fullsized":"2024-09-10T22:38:41.591415+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:46.906395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590974+0000","last_change":"2024-09-10T22:38:40.613412+0000","last_active":"2024-09-10T22:38:41.590974+0000","last_peered":"2024-09-10T22:38:41.590974+0000","last_clean":"2024-09-10T22:38:41.590974+0000","last_became_active":"2024-09-10T22:38:40.613122+0000","last_became_peered":"2024-09-10T22:38:40.613122+0000","last_unstale":"2024-09-10T22:38:41.590974+0000","last_undegraded":"2024-09-10T22:38:41.590974+0000","last_fullsized":"2024-09-10T22:38:41.590974+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:13.539196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.617741+0000","last_change":"2024-09-10T22:38:40.615868+0000","last_active":"2024-09-10T22:38:40.617741+0000","last_peered":"2024-09-10T22:38:40.617741+0000","last_clean":"2024-09-10T22:38:40.617741+0000","last_became_active":"2024-09-10T22:38:40.615487+0000","last_became_peered":"2024-09-10T22:38:40.615487+0000","last_unstale":"2024-09-10T22:38:40.617741+0000","last_undegraded":"2024-09-10T22:38:40.617741+0000","last_fullsized":"2024-09-10T22:38:40.617741+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:40.335621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.620627+0000","last_change":"2024-09-10T22:38:40.612550+0000","last_active":"2024-09-10T22:38:40.620627+0000","last_peered":"2024-09-10T22:38:40.620627+0000","last_clean":"2024-09-10T22:38:40.620627+0000","last_became_active":"2024-09-10T22:38:40.612245+0000","last_became_peered":"2024-09-10T22:38:40.612245+0000","last_unstale":"2024-09-10T22:38:40.620627+0000","last_undegraded":"2024-09-10T22:38:40.620627+0000","last_fullsized":"2024-09-10T22:38:40.620627+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:08.583512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.610091+0000","last_change":"2024-09-10T22:38:40.607316+0000","last_active":"2024-09-10T22:38:40.610091+0000","last_peered":"2024-09-10T22:38:40.610091+0000","last_clean":"2024-09-10T22:38:40.610091+0000","last_became_active":"2024-09-10T22:38:40.606989+0000","last_became_peered":"2024-09-10T22:38:40.606989+0000","last_unstale":"2024-09-10T22:38:40.610091+0000","last_undegraded":"2024-09-10T22:38:40.610091+0000","last_fullsized":"2024-09-10T22:38:40.610091+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:08.970351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616066+0000","last_change":"2024-09-10T22:38:40.608365+0000","last_active":"2024-09-10T22:38:40.616066+0000","last_peered":"2024-09-10T22:38:40.616066+0000","last_clean":"2024-09-10T22:38:40.616066+0000","last_became_active":"2024-09-10T22:38:40.607758+0000","last_became_peered":"2024-09-10T22:38:40.607758+0000","last_unstale":"2024-09-10T22:38:40.616066+0000","last_undegraded":"2024-09-10T22:38:40.616066+0000","last_fullsized":"2024-09-10T22:38:40.616066+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:20.283263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608766+0000","last_change":"2024-09-10T22:38:40.607815+0000","last_active":"2024-09-10T22:38:40.608766+0000","last_peered":"2024-09-10T22:38:40.608766+0000","last_clean":"2024-09-10T22:38:40.608766+0000","last_became_active":"2024-09-10T22:38:40.607549+0000","last_became_peered":"2024-09-10T22:38:40.607549+0000","last_unstale":"2024-09-10T22:38:40.608766+0000","last_undegraded":"2024-09-10T22:38:40.608766+0000","last_fullsized":"2024-09-10T22:38:40.608766+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:52.827052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607983+0000","last_change":"2024-09-10T22:38:40.606116+0000","last_active":"2024-09-10T22:38:40.607983+0000","last_peered":"2024-09-10T22:38:40.607983+0000","last_clean":"2024-09-10T22:38:40.607983+0000","last_became_active":"2024-09-10T22:38:40.605765+0000","last_became_peered":"2024-09-10T22:38:40.605765+0000","last_unstale":"2024-09-10T22:38:40.607983+0000","last_undegraded":"2024-09-10T22:38:40.607983+0000","last_fullsized":"2024-09-10T22:38:40.607983+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:41.024009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607798+0000","last_change":"2024-09-10T22:38:40.603536+0000","last_active":"2024-09-10T22:38:40.607798+0000","last_peered":"2024-09-10T22:38:40.607798+0000","last_clean":"2024-09-10T22:38:40.607798+0000","last_became_active":"2024-09-10T22:38:40.603242+0000","last_became_peered":"2024-09-10T22:38:40.603242+0000","last_unstale":"2024-09-10T22:38:40.607798+0000","last_undegraded":"2024-09-10T22:38:40.607798+0000","last_fullsized":"2024-09-10T22:38:40.607798+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:09.989910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611088+0000","last_change":"2024-09-10T22:38:40.606799+0000","last_active":"2024-09-10T22:38:40.611088+0000","last_peered":"2024-09-10T22:38:40.611088+0000","last_clean":"2024-09-10T22:38:40.611088+0000","last_became_active":"2024-09-10T22:38:40.606553+0000","last_became_peered":"2024-09-10T22:38:40.606553+0000","last_unstale":"2024-09-10T22:38:40.611088+0000","last_undegraded":"2024-09-10T22:38:40.611088+0000","last_fullsized":"2024-09-10T22:38:40.611088+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:53.338610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590940+0000","last_change":"2024-09-10T22:38:40.596080+0000","last_active":"2024-09-10T22:38:41.590940+0000","last_peered":"2024-09-10T22:38:41.590940+0000","last_clean":"2024-09-10T22:38:41.590940+0000","last_became_active":"2024-09-10T22:38:40.595795+0000","last_became_peered":"2024-09-10T22:38:40.595795+0000","last_unstale":"2024-09-10T22:38:41.590940+0000","last_undegraded":"2024-09-10T22:38:41.590940+0000","last_fullsized":"2024-09-10T22:38:41.590940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:00:55.090423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591182+0000","last_change":"2024-09-10T22:38:40.594835+0000","last_active":"2024-09-10T22:38:41.591182+0000","last_peered":"2024-09-10T22:38:41.591182+0000","last_clean":"2024-09-10T22:38:41.591182+0000","last_became_active":"2024-09-10T22:38:40.594533+0000","last_became_peered":"2024-09-10T22:38:40.594533+0000","last_unstale":"2024-09-10T22:38:41.591182+0000","last_undegraded":"2024-09-10T22:38:41.591182+0000","last_fullsized":"2024-09-10T22:38:41.591182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:12.084168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607849+0000","last_change":"2024-09-10T22:38:40.604895+0000","last_active":"2024-09-10T22:38:40.607849+0000","last_peered":"2024-09-10T22:38:40.607849+0000","last_clean":"2024-09-10T22:38:40.607849+0000","last_became_active":"2024-09-10T22:38:40.604335+0000","last_became_peered":"2024-09-10T22:38:40.604335+0000","last_unstale":"2024-09-10T22:38:40.607849+0000","last_undegraded":"2024-09-10T22:38:40.607849+0000","last_fullsized":"2024-09-10T22:38:40.607849+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:11.776292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591435+0000","last_change":"2024-09-10T22:38:40.597022+0000","last_active":"2024-09-10T22:38:41.591435+0000","last_peered":"2024-09-10T22:38:41.591435+0000","last_clean":"2024-09-10T22:38:41.591435+0000","last_became_active":"2024-09-10T22:38:40.596693+0000","last_became_peered":"2024-09-10T22:38:40.596693+0000","last_unstale":"2024-09-10T22:38:41.591435+0000","last_undegraded":"2024-09-10T22:38:41.591435+0000","last_fullsized":"2024-09-10T22:38:41.591435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:42.085973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607678+0000","last_change":"2024-09-10T22:38:40.603896+0000","last_active":"2024-09-10T22:38:40.607678+0000","last_peered":"2024-09-10T22:38:40.607678+0000","last_clean":"2024-09-10T22:38:40.607678+0000","last_became_active":"2024-09-10T22:38:40.602970+0000","last_became_peered":"2024-09-10T22:38:40.602970+0000","last_unstale":"2024-09-10T22:38:40.607678+0000","last_undegraded":"2024-09-10T22:38:40.607678+0000","last_fullsized":"2024-09-10T22:38:40.607678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:45.095703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602395+0000","last_change":"2024-09-10T22:38:40.600178+0000","last_active":"2024-09-10T22:38:40.602395+0000","last_peered":"2024-09-10T22:38:40.602395+0000","last_clean":"2024-09-10T22:38:40.602395+0000","last_became_active":"2024-09-10T22:38:40.599906+0000","last_became_peered":"2024-09-10T22:38:40.599906+0000","last_unstale":"2024-09-10T22:38:40.602395+0000","last_undegraded":"2024-09-10T22:38:40.602395+0000","last_fullsized":"2024-09-10T22:38:40.602395+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:24.232203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611393+0000","last_change":"2024-09-10T22:38:40.609977+0000","last_active":"2024-09-10T22:38:40.611393+0000","last_peered":"2024-09-10T22:38:40.611393+0000","last_clean":"2024-09-10T22:38:40.611393+0000","last_became_active":"2024-09-10T22:38:40.609446+0000","last_became_peered":"2024-09-10T22:38:40.609446+0000","last_unstale":"2024-09-10T22:38:40.611393+0000","last_undegraded":"2024-09-10T22:38:40.611393+0000","last_fullsized":"2024-09-10T22:38:40.611393+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:44.400517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.625805+0000","last_change":"2024-09-10T22:38:39.605644+0000","last_active":"2024-09-10T22:38:40.625805+0000","last_peered":"2024-09-10T22:38:40.625805+0000","last_clean":"2024-09-10T22:38:40.625805+0000","last_became_active":"2024-09-10T22:38:39.605193+0000","last_became_peered":"2024-09-10T22:38:39.605193+0000","last_unstale":"2024-09-10T22:38:40.625805+0000","last_undegraded":"2024-09-10T22:38:40.625805+0000","last_fullsized":"2024-09-10T22:38:40.625805+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:10.706373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.605313+0000","last_change":"2024-09-10T22:38:40.602604+0000","last_active":"2024-09-10T22:38:40.605313+0000","last_peered":"2024-09-10T22:38:40.605313+0000","last_clean":"2024-09-10T22:38:40.605313+0000","last_became_active":"2024-09-10T22:38:40.602299+0000","last_became_peered":"2024-09-10T22:38:40.602299+0000","last_unstale":"2024-09-10T22:38:40.605313+0000","last_undegraded":"2024-09-10T22:38:40.605313+0000","last_fullsized":"2024-09-10T22:38:40.605313+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:04.646635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591305+0000","last_change":"2024-09-10T22:38:39.594655+0000","last_active":"2024-09-10T22:38:41.591305+0000","last_peered":"2024-09-10T22:38:41.591305+0000","last_clean":"2024-09-10T22:38:41.591305+0000","last_became_active":"2024-09-10T22:38:39.594350+0000","last_became_peered":"2024-09-10T22:38:39.594350+0000","last_unstale":"2024-09-10T22:38:41.591305+0000","last_undegraded":"2024-09-10T22:38:41.591305+0000","last_fullsized":"2024-09-10T22:38:41.591305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:19.280955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591548+0000","last_change":"2024-09-10T22:38:40.606263+0000","last_active":"2024-09-10T22:38:41.591548+0000","last_peered":"2024-09-10T22:38:41.591548+0000","last_clean":"2024-09-10T22:38:41.591548+0000","last_became_active":"2024-09-10T22:38:40.605955+0000","last_became_peered":"2024-09-10T22:38:40.605955+0000","last_unstale":"2024-09-10T22:38:41.591548+0000","last_undegraded":"2024-09-10T22:38:41.591548+0000","last_fullsized":"2024-09-10T22:38:41.591548+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:02.571771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.610981+0000","last_change":"2024-09-10T22:38:39.599700+0000","last_active":"2024-09-10T22:38:40.610981+0000","last_peered":"2024-09-10T22:38:40.610981+0000","last_clean":"2024-09-10T22:38:40.610981+0000","last_became_active":"2024-09-10T22:38:39.599372+0000","last_became_peered":"2024-09-10T22:38:39.599372+0000","last_unstale":"2024-09-10T22:38:40.610981+0000","last_undegraded":"2024-09-10T22:38:40.610981+0000","last_fullsized":"2024-09-10T22:38:40.610981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:32.967428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611041+0000","last_change":"2024-09-10T22:38:40.606245+0000","last_active":"2024-09-10T22:38:40.611041+0000","last_peered":"2024-09-10T22:38:40.611041+0000","last_clean":"2024-09-10T22:38:40.611041+0000","last_became_active":"2024-09-10T22:38:40.605911+0000","last_became_peered":"2024-09-10T22:38:40.605911+0000","last_unstale":"2024-09-10T22:38:40.611041+0000","last_undegraded":"2024-09-10T22:38:40.611041+0000","last_fullsized":"2024-09-10T22:38:40.611041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:37.382726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.618164+0000","last_change":"2024-09-10T22:38:39.602630+0000","last_active":"2024-09-10T22:38:40.618164+0000","last_peered":"2024-09-10T22:38:40.618164+0000","last_clean":"2024-09-10T22:38:40.618164+0000","last_became_active":"2024-09-10T22:38:39.602348+0000","last_became_peered":"2024-09-10T22:38:39.602348+0000","last_unstale":"2024-09-10T22:38:40.618164+0000","last_undegraded":"2024-09-10T22:38:40.618164+0000","last_fullsized":"2024-09-10T22:38:40.618164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:02.672307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591657+0000","last_change":"2024-09-10T22:38:40.607120+0000","last_active":"2024-09-10T22:38:41.591657+0000","last_peered":"2024-09-10T22:38:41.591657+0000","last_clean":"2024-09-10T22:38:41.591657+0000","last_became_active":"2024-09-10T22:38:40.606811+0000","last_became_peered":"2024-09-10T22:38:40.606811+0000","last_unstale":"2024-09-10T22:38:41.591657+0000","last_undegraded":"2024-09-10T22:38:41.591657+0000","last_fullsized":"2024-09-10T22:38:41.591657+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:39.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.625673+0000","last_change":"2024-09-10T22:38:39.605598+0000","last_active":"2024-09-10T22:38:40.625673+0000","last_peered":"2024-09-10T22:38:40.625673+0000","last_clean":"2024-09-10T22:38:40.625673+0000","last_became_active":"2024-09-10T22:38:39.605276+0000","last_became_peered":"2024-09-10T22:38:39.605276+0000","last_unstale":"2024-09-10T22:38:40.625673+0000","last_undegraded":"2024-09-10T22:38:40.625673+0000","last_fullsized":"2024-09-10T22:38:40.625673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:21.150701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616040+0000","last_change":"2024-09-10T22:38:40.609038+0000","last_active":"2024-09-10T22:38:40.616040+0000","last_peered":"2024-09-10T22:38:40.616040+0000","last_clean":"2024-09-10T22:38:40.616040+0000","last_became_active":"2024-09-10T22:38:40.608759+0000","last_became_peered":"2024-09-10T22:38:40.608759+0000","last_unstale":"2024-09-10T22:38:40.616040+0000","last_undegraded":"2024-09-10T22:38:40.616040+0000","last_fullsized":"2024-09-10T22:38:40.616040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:41.939124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591792+0000","last_change":"2024-09-10T22:38:39.599770+0000","last_active":"2024-09-10T22:38:41.591792+0000","last_peered":"2024-09-10T22:38:41.591792+0000","last_clean":"2024-09-10T22:38:41.591792+0000","last_became_active":"2024-09-10T22:38:39.599481+0000","last_became_peered":"2024-09-10T22:38:39.599481+0000","last_unstale":"2024-09-10T22:38:41.591792+0000","last_undegraded":"2024-09-10T22:38:41.591792+0000","last_fullsized":"2024-09-10T22:38:41.591792+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:40.936816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616555+0000","last_change":"2024-09-10T22:38:40.611882+0000","last_active":"2024-09-10T22:38:40.616555+0000","last_peered":"2024-09-10T22:38:40.616555+0000","last_clean":"2024-09-10T22:38:40.616555+0000","last_became_active":"2024-09-10T22:38:40.608025+0000","last_became_peered":"2024-09-10T22:38:40.608025+0000","last_unstale":"2024-09-10T22:38:40.616555+0000","last_undegraded":"2024-09-10T22:38:40.616555+0000","last_fullsized":"2024-09-10T22:38:40.616555+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:06.774831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590815+0000","last_change":"2024-09-10T22:38:39.605345+0000","last_active":"2024-09-10T22:38:41.590815+0000","last_peered":"2024-09-10T22:38:41.590815+0000","last_clean":"2024-09-10T22:38:41.590815+0000","last_became_active":"2024-09-10T22:38:39.605027+0000","last_became_peered":"2024-09-10T22:38:39.605027+0000","last_unstale":"2024-09-10T22:38:41.590815+0000","last_undegraded":"2024-09-10T22:38:41.590815+0000","last_fullsized":"2024-09-10T22:38:41.590815+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:13.394525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608738+0000","last_change":"2024-09-10T22:38:40.607144+0000","last_active":"2024-09-10T22:38:40.608738+0000","last_peered":"2024-09-10T22:38:40.608738+0000","last_clean":"2024-09-10T22:38:40.608738+0000","last_became_active":"2024-09-10T22:38:40.605350+0000","last_became_peered":"2024-09-10T22:38:40.605350+0000","last_unstale":"2024-09-10T22:38:40.608738+0000","last_undegraded":"2024-09-10T22:38:40.608738+0000","last_fullsized":"2024-09-10T22:38:40.608738+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:07.780094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602377+0000","last_change":"2024-09-10T22:38:39.608819+0000","last_active":"2024-09-10T22:38:40.602377+0000","last_peered":"2024-09-10T22:38:40.602377+0000","last_clean":"2024-09-10T22:38:40.602377+0000","last_became_active":"2024-09-10T22:38:39.608486+0000","last_became_peered":"2024-09-10T22:38:39.608486+0000","last_unstale":"2024-09-10T22:38:40.602377+0000","last_undegraded":"2024-09-10T22:38:40.602377+0000","last_fullsized":"2024-09-10T22:38:40.602377+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:49.482624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591181+0000","last_change":"2024-09-10T22:38:40.612734+0000","last_active":"2024-09-10T22:38:41.591181+0000","last_peered":"2024-09-10T22:38:41.591181+0000","last_clean":"2024-09-10T22:38:41.591181+0000","last_became_active":"2024-09-10T22:38:40.612456+0000","last_became_peered":"2024-09-10T22:38:40.612456+0000","last_unstale":"2024-09-10T22:38:41.591181+0000","last_undegraded":"2024-09-10T22:38:41.591181+0000","last_fullsized":"2024-09-10T22:38:41.591181+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:02.766716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591612+0000","last_change":"2024-09-10T22:38:39.596794+0000","last_active":"2024-09-10T22:38:41.591612+0000","last_peered":"2024-09-10T22:38:41.591612+0000","last_clean":"2024-09-10T22:38:41.591612+0000","last_became_active":"2024-09-10T22:38:39.596516+0000","last_became_peered":"2024-09-10T22:38:39.596516+0000","last_unstale":"2024-09-10T22:38:41.591612+0000","last_undegraded":"2024-09-10T22:38:41.591612+0000","last_fullsized":"2024-09-10T22:38:41.591612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:33.201448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.604972+0000","last_change":"2024-09-10T22:38:40.598896+0000","last_active":"2024-09-10T22:38:40.604972+0000","last_peered":"2024-09-10T22:38:40.604972+0000","last_clean":"2024-09-10T22:38:40.604972+0000","last_became_active":"2024-09-10T22:38:40.598621+0000","last_became_peered":"2024-09-10T22:38:40.598621+0000","last_unstale":"2024-09-10T22:38:40.604972+0000","last_undegraded":"2024-09-10T22:38:40.604972+0000","last_fullsized":"2024-09-10T22:38:40.604972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:13.539196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.603841+0000","last_change":"2024-09-10T22:38:39.609051+0000","last_active":"2024-09-10T22:38:40.603841+0000","last_peered":"2024-09-10T22:38:40.603841+0000","last_clean":"2024-09-10T22:38:40.603841+0000","last_became_active":"2024-09-10T22:38:39.608768+0000","last_became_peered":"2024-09-10T22:38:39.608768+0000","last_unstale":"2024-09-10T22:38:40.603841+0000","last_undegraded":"2024-09-10T22:38:40.603841+0000","last_fullsized":"2024-09-10T22:38:40.603841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:37.387484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616260+0000","last_change":"2024-09-10T22:38:40.610099+0000","last_active":"2024-09-10T22:38:40.616260+0000","last_peered":"2024-09-10T22:38:40.616260+0000","last_clean":"2024-09-10T22:38:40.616260+0000","last_became_active":"2024-09-10T22:38:40.609557+0000","last_became_peered":"2024-09-10T22:38:40.609557+0000","last_unstale":"2024-09-10T22:38:40.616260+0000","last_undegraded":"2024-09-10T22:38:40.616260+0000","last_fullsized":"2024-09-10T22:38:40.616260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:42.245161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.617912+0000","last_change":"2024-09-10T22:38:39.600819+0000","last_active":"2024-09-10T22:38:40.617912+0000","last_peered":"2024-09-10T22:38:40.617912+0000","last_clean":"2024-09-10T22:38:40.617912+0000","last_became_active":"2024-09-10T22:38:39.600215+0000","last_became_peered":"2024-09-10T22:38:39.600215+0000","last_unstale":"2024-09-10T22:38:40.617912+0000","last_undegraded":"2024-09-10T22:38:40.617912+0000","last_fullsized":"2024-09-10T22:38:40.617912+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:43.322365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.604757+0000","last_change":"2024-09-10T22:38:40.601356+0000","last_active":"2024-09-10T22:38:40.604757+0000","last_peered":"2024-09-10T22:38:40.604757+0000","last_clean":"2024-09-10T22:38:40.604757+0000","last_became_active":"2024-09-10T22:38:40.601073+0000","last_became_peered":"2024-09-10T22:38:40.601073+0000","last_unstale":"2024-09-10T22:38:40.604757+0000","last_undegraded":"2024-09-10T22:38:40.604757+0000","last_fullsized":"2024-09-10T22:38:40.604757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:46.906395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":19,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.627138+0000","last_change":"2024-09-10T22:38:39.593820+0000","last_active":"2024-09-10T22:38:40.627138+0000","last_peered":"2024-09-10T22:38:40.627138+0000","last_clean":"2024-09-10T22:38:40.627138+0000","last_became_active":"2024-09-10T22:38:39.593642+0000","last_became_peered":"2024-09-10T22:38:39.593642+0000","last_unstale":"2024-09-10T22:38:40.627138+0000","last_undegraded":"2024-09-10T22:38:40.627138+0000","last_fullsized":"2024-09-10T22:38:40.627138+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.216783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.617782+0000","last_change":"2024-09-10T22:38:40.616137+0000","last_active":"2024-09-10T22:38:40.617782+0000","last_peered":"2024-09-10T22:38:40.617782+0000","last_clean":"2024-09-10T22:38:40.617782+0000","last_became_active":"2024-09-10T22:38:40.615880+0000","last_became_peered":"2024-09-10T22:38:40.615880+0000","last_unstale":"2024-09-10T22:38:40.617782+0000","last_undegraded":"2024-09-10T22:38:40.617782+0000","last_fullsized":"2024-09-10T22:38:40.617782+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:02.924596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.620950+0000","last_change":"2024-09-10T22:38:39.609121+0000","last_active":"2024-09-10T22:38:40.620950+0000","last_peered":"2024-09-10T22:38:40.620950+0000","last_clean":"2024-09-10T22:38:40.620950+0000","last_became_active":"2024-09-10T22:38:39.608803+0000","last_became_peered":"2024-09-10T22:38:39.608803+0000","last_unstale":"2024-09-10T22:38:40.620950+0000","last_undegraded":"2024-09-10T22:38:40.620950+0000","last_fullsized":"2024-09-10T22:38:40.620950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:34.687683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.616014+0000","last_change":"2024-09-10T22:38:40.607818+0000","last_active":"2024-09-10T22:38:40.616014+0000","last_peered":"2024-09-10T22:38:40.616014+0000","last_clean":"2024-09-10T22:38:40.616014+0000","last_became_active":"2024-09-10T22:38:40.605884+0000","last_became_peered":"2024-09-10T22:38:40.605884+0000","last_unstale":"2024-09-10T22:38:40.616014+0000","last_undegraded":"2024-09-10T22:38:40.616014+0000","last_fullsized":"2024-09-10T22:38:40.616014+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:34.203756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593296+0000","last_change":"2024-09-10T22:38:39.599483+0000","last_active":"2024-09-10T22:38:40.593296+0000","last_peered":"2024-09-10T22:38:40.593296+0000","last_clean":"2024-09-10T22:38:40.593296+0000","last_became_active":"2024-09-10T22:38:39.599229+0000","last_became_peered":"2024-09-10T22:38:39.599229+0000","last_unstale":"2024-09-10T22:38:40.593296+0000","last_undegraded":"2024-09-10T22:38:40.593296+0000","last_fullsized":"2024-09-10T22:38:40.593296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:51.704795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607582+0000","last_change":"2024-09-10T22:38:40.605081+0000","last_active":"2024-09-10T22:38:40.607582+0000","last_peered":"2024-09-10T22:38:40.607582+0000","last_clean":"2024-09-10T22:38:40.607582+0000","last_became_active":"2024-09-10T22:38:40.604772+0000","last_became_peered":"2024-09-10T22:38:40.604772+0000","last_unstale":"2024-09-10T22:38:40.607582+0000","last_undegraded":"2024-09-10T22:38:40.607582+0000","last_fullsized":"2024-09-10T22:38:40.607582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:00.803195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.596085+0000","last_change":"2024-09-10T22:38:39.609549+0000","last_active":"2024-09-10T22:38:40.596085+0000","last_peered":"2024-09-10T22:38:40.596085+0000","last_clean":"2024-09-10T22:38:40.596085+0000","last_became_active":"2024-09-10T22:38:39.609065+0000","last_became_peered":"2024-09-10T22:38:39.609065+0000","last_unstale":"2024-09-10T22:38:40.596085+0000","last_undegraded":"2024-09-10T22:38:40.596085+0000","last_fullsized":"2024-09-10T22:38:40.596085+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:15.236028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.609911+0000","last_change":"2024-09-10T22:38:40.606321+0000","last_active":"2024-09-10T22:38:40.609911+0000","last_peered":"2024-09-10T22:38:40.609911+0000","last_clean":"2024-09-10T22:38:40.609911+0000","last_became_active":"2024-09-10T22:38:40.605852+0000","last_became_peered":"2024-09-10T22:38:40.605852+0000","last_unstale":"2024-09-10T22:38:40.609911+0000","last_undegraded":"2024-09-10T22:38:40.609911+0000","last_fullsized":"2024-09-10T22:38:40.609911+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:39.680834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.623199+0000","last_change":"2024-09-10T22:38:39.608173+0000","last_active":"2024-09-10T22:38:40.623199+0000","last_peered":"2024-09-10T22:38:40.623199+0000","last_clean":"2024-09-10T22:38:40.623199+0000","last_became_active":"2024-09-10T22:38:39.607830+0000","last_became_peered":"2024-09-10T22:38:39.607830+0000","last_unstale":"2024-09-10T22:38:40.623199+0000","last_undegraded":"2024-09-10T22:38:40.623199+0000","last_fullsized":"2024-09-10T22:38:40.623199+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:39.333312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.601704+0000","last_change":"2024-09-10T22:38:40.593482+0000","last_active":"2024-09-10T22:38:40.601704+0000","last_peered":"2024-09-10T22:38:40.601704+0000","last_clean":"2024-09-10T22:38:40.601704+0000","last_became_active":"2024-09-10T22:38:40.593215+0000","last_became_peered":"2024-09-10T22:38:40.593215+0000","last_unstale":"2024-09-10T22:38:40.601704+0000","last_undegraded":"2024-09-10T22:38:40.601704+0000","last_fullsized":"2024-09-10T22:38:40.601704+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:29.594498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.626090+0000","last_change":"2024-09-10T22:38:39.607157+0000","last_active":"2024-09-10T22:38:40.626090+0000","last_peered":"2024-09-10T22:38:40.626090+0000","last_clean":"2024-09-10T22:38:40.626090+0000","last_became_active":"2024-09-10T22:38:39.606835+0000","last_became_peered":"2024-09-10T22:38:39.606835+0000","last_unstale":"2024-09-10T22:38:40.626090+0000","last_undegraded":"2024-09-10T22:38:40.626090+0000","last_fullsized":"2024-09-10T22:38:40.626090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:05.761073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.618297+0000","last_change":"2024-09-10T22:38:40.612589+0000","last_active":"2024-09-10T22:38:40.618297+0000","last_peered":"2024-09-10T22:38:40.618297+0000","last_clean":"2024-09-10T22:38:40.618297+0000","last_became_active":"2024-09-10T22:38:40.612311+0000","last_became_peered":"2024-09-10T22:38:40.612311+0000","last_unstale":"2024-09-10T22:38:40.618297+0000","last_undegraded":"2024-09-10T22:38:40.618297+0000","last_fullsized":"2024-09-10T22:38:40.618297+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:59.412309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591422+0000","last_change":"2024-09-10T22:38:39.604556+0000","last_active":"2024-09-10T22:38:41.591422+0000","last_peered":"2024-09-10T22:38:41.591422+0000","last_clean":"2024-09-10T22:38:41.591422+0000","last_became_active":"2024-09-10T22:38:39.604119+0000","last_became_peered":"2024-09-10T22:38:39.604119+0000","last_unstale":"2024-09-10T22:38:41.591422+0000","last_undegraded":"2024-09-10T22:38:41.591422+0000","last_fullsized":"2024-09-10T22:38:41.591422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:02.516476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.611256+0000","last_change":"2024-09-10T22:38:40.609572+0000","last_active":"2024-09-10T22:38:40.611256+0000","last_peered":"2024-09-10T22:38:40.611256+0000","last_clean":"2024-09-10T22:38:40.611256+0000","last_became_active":"2024-09-10T22:38:40.608929+0000","last_became_peered":"2024-09-10T22:38:40.608929+0000","last_unstale":"2024-09-10T22:38:40.611256+0000","last_undegraded":"2024-09-10T22:38:40.611256+0000","last_fullsized":"2024-09-10T22:38:40.611256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:54.564665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591125+0000","last_change":"2024-09-10T22:38:39.605022+0000","last_active":"2024-09-10T22:38:41.591125+0000","last_peered":"2024-09-10T22:38:41.591125+0000","last_clean":"2024-09-10T22:38:41.591125+0000","last_became_active":"2024-09-10T22:38:39.604688+0000","last_became_peered":"2024-09-10T22:38:39.604688+0000","last_unstale":"2024-09-10T22:38:41.591125+0000","last_undegraded":"2024-09-10T22:38:41.591125+0000","last_fullsized":"2024-09-10T22:38:41.591125+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:55.144203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607789+0000","last_change":"2024-09-10T22:38:40.604783+0000","last_active":"2024-09-10T22:38:40.607789+0000","last_peered":"2024-09-10T22:38:40.607789+0000","last_clean":"2024-09-10T22:38:40.607789+0000","last_became_active":"2024-09-10T22:38:40.604447+0000","last_became_peered":"2024-09-10T22:38:40.604447+0000","last_unstale":"2024-09-10T22:38:40.607789+0000","last_undegraded":"2024-09-10T22:38:40.607789+0000","last_fullsized":"2024-09-10T22:38:40.607789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:20.249674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.596706+0000","last_change":"2024-09-10T22:38:39.606706+0000","last_active":"2024-09-10T22:38:40.596706+0000","last_peered":"2024-09-10T22:38:40.596706+0000","last_clean":"2024-09-10T22:38:40.596706+0000","last_became_active":"2024-09-10T22:38:39.606414+0000","last_became_peered":"2024-09-10T22:38:39.606414+0000","last_unstale":"2024-09-10T22:38:40.596706+0000","last_undegraded":"2024-09-10T22:38:40.596706+0000","last_fullsized":"2024-09-10T22:38:40.596706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:15.835251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.614056+0000","last_change":"2024-09-10T22:38:40.610133+0000","last_active":"2024-09-10T22:38:40.614056+0000","last_peered":"2024-09-10T22:38:40.614056+0000","last_clean":"2024-09-10T22:38:40.614056+0000","last_became_active":"2024-09-10T22:38:40.609526+0000","last_became_peered":"2024-09-10T22:38:40.609526+0000","last_unstale":"2024-09-10T22:38:40.614056+0000","last_undegraded":"2024-09-10T22:38:40.614056+0000","last_fullsized":"2024-09-10T22:38:40.614056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:08.333001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.625759+0000","last_change":"2024-09-10T22:38:39.603931+0000","last_active":"2024-09-10T22:38:40.625759+0000","last_peered":"2024-09-10T22:38:40.625759+0000","last_clean":"2024-09-10T22:38:40.625759+0000","last_became_active":"2024-09-10T22:38:39.603658+0000","last_became_peered":"2024-09-10T22:38:39.603658+0000","last_unstale":"2024-09-10T22:38:40.625759+0000","last_undegraded":"2024-09-10T22:38:40.625759+0000","last_fullsized":"2024-09-10T22:38:40.625759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:20:41.825217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.609856+0000","last_change":"2024-09-10T22:38:40.605841+0000","last_active":"2024-09-10T22:38:40.609856+0000","last_peered":"2024-09-10T22:38:40.609856+0000","last_clean":"2024-09-10T22:38:40.609856+0000","last_became_active":"2024-09-10T22:38:40.605467+0000","last_became_peered":"2024-09-10T22:38:40.605467+0000","last_unstale":"2024-09-10T22:38:40.609856+0000","last_undegraded":"2024-09-10T22:38:40.609856+0000","last_fullsized":"2024-09-10T22:38:40.609856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:00:40.816546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590789+0000","last_change":"2024-09-10T22:38:39.598825+0000","last_active":"2024-09-10T22:38:41.590789+0000","last_peered":"2024-09-10T22:38:41.590789+0000","last_clean":"2024-09-10T22:38:41.590789+0000","last_became_active":"2024-09-10T22:38:39.598473+0000","last_became_peered":"2024-09-10T22:38:39.598473+0000","last_unstale":"2024-09-10T22:38:41.590789+0000","last_undegraded":"2024-09-10T22:38:41.590789+0000","last_fullsized":"2024-09-10T22:38:41.590789+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:04.427415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602370+0000","last_change":"2024-09-10T22:38:40.599523+0000","last_active":"2024-09-10T22:38:40.602370+0000","last_peered":"2024-09-10T22:38:40.602370+0000","last_clean":"2024-09-10T22:38:40.602370+0000","last_became_active":"2024-09-10T22:38:40.599241+0000","last_became_peered":"2024-09-10T22:38:40.599241+0000","last_unstale":"2024-09-10T22:38:40.602370+0000","last_undegraded":"2024-09-10T22:38:40.602370+0000","last_fullsized":"2024-09-10T22:38:40.602370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:14.083091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.599230+0000","last_change":"2024-09-10T22:38:39.608091+0000","last_active":"2024-09-10T22:38:40.599230+0000","last_peered":"2024-09-10T22:38:40.599230+0000","last_clean":"2024-09-10T22:38:40.599230+0000","last_became_active":"2024-09-10T22:38:39.607826+0000","last_became_peered":"2024-09-10T22:38:39.607826+0000","last_unstale":"2024-09-10T22:38:40.599230+0000","last_undegraded":"2024-09-10T22:38:40.599230+0000","last_fullsized":"2024-09-10T22:38:40.599230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:51.824744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.620459+0000","last_change":"2024-09-10T22:38:40.614672+0000","last_active":"2024-09-10T22:38:40.620459+0000","last_peered":"2024-09-10T22:38:40.620459+0000","last_clean":"2024-09-10T22:38:40.620459+0000","last_became_active":"2024-09-10T22:38:40.614357+0000","last_became_peered":"2024-09-10T22:38:40.614357+0000","last_unstale":"2024-09-10T22:38:40.620459+0000","last_undegraded":"2024-09-10T22:38:40.620459+0000","last_fullsized":"2024-09-10T22:38:40.620459+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:52.707103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593649+0000","last_change":"2024-09-10T22:38:39.602385+0000","last_active":"2024-09-10T22:38:40.593649+0000","last_peered":"2024-09-10T22:38:40.593649+0000","last_clean":"2024-09-10T22:38:40.593649+0000","last_became_active":"2024-09-10T22:38:39.601901+0000","last_became_peered":"2024-09-10T22:38:39.601901+0000","last_unstale":"2024-09-10T22:38:40.593649+0000","last_undegraded":"2024-09-10T22:38:40.593649+0000","last_fullsized":"2024-09-10T22:38:40.593649+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:48.909028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.613680+0000","last_change":"2024-09-10T22:38:40.612463+0000","last_active":"2024-09-10T22:38:40.613680+0000","last_peered":"2024-09-10T22:38:40.613680+0000","last_clean":"2024-09-10T22:38:40.613680+0000","last_became_active":"2024-09-10T22:38:40.612148+0000","last_became_peered":"2024-09-10T22:38:40.612148+0000","last_unstale":"2024-09-10T22:38:40.613680+0000","last_undegraded":"2024-09-10T22:38:40.613680+0000","last_fullsized":"2024-09-10T22:38:40.613680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:50.484932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.595666+0000","last_change":"2024-09-10T22:38:39.605995+0000","last_active":"2024-09-10T22:38:40.595666+0000","last_peered":"2024-09-10T22:38:40.595666+0000","last_clean":"2024-09-10T22:38:40.595666+0000","last_became_active":"2024-09-10T22:38:39.604806+0000","last_became_peered":"2024-09-10T22:38:39.604806+0000","last_unstale":"2024-09-10T22:38:40.595666+0000","last_undegraded":"2024-09-10T22:38:40.595666+0000","last_fullsized":"2024-09-10T22:38:40.595666+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:15.502820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608912+0000","last_change":"2024-09-10T22:38:40.602467+0000","last_active":"2024-09-10T22:38:40.608912+0000","last_peered":"2024-09-10T22:38:40.608912+0000","last_clean":"2024-09-10T22:38:40.608912+0000","last_became_active":"2024-09-10T22:38:40.602131+0000","last_became_peered":"2024-09-10T22:38:40.602131+0000","last_unstale":"2024-09-10T22:38:40.608912+0000","last_undegraded":"2024-09-10T22:38:40.608912+0000","last_fullsized":"2024-09-10T22:38:40.608912+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:52.302762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.594567+0000","last_change":"2024-09-10T22:38:39.600489+0000","last_active":"2024-09-10T22:38:40.594567+0000","last_peered":"2024-09-10T22:38:40.594567+0000","last_clean":"2024-09-10T22:38:40.594567+0000","last_became_active":"2024-09-10T22:38:39.599888+0000","last_became_peered":"2024-09-10T22:38:39.599888+0000","last_unstale":"2024-09-10T22:38:40.594567+0000","last_undegraded":"2024-09-10T22:38:40.594567+0000","last_fullsized":"2024-09-10T22:38:40.594567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:47.450705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.605550+0000","last_change":"2024-09-10T22:38:40.603701+0000","last_active":"2024-09-10T22:38:40.605550+0000","last_peered":"2024-09-10T22:38:40.605550+0000","last_clean":"2024-09-10T22:38:40.605550+0000","last_became_active":"2024-09-10T22:38:40.603445+0000","last_became_peered":"2024-09-10T22:38:40.603445+0000","last_unstale":"2024-09-10T22:38:40.605550+0000","last_undegraded":"2024-09-10T22:38:40.605550+0000","last_fullsized":"2024-09-10T22:38:40.605550+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:42.085973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.597906+0000","last_change":"2024-09-10T22:38:39.603987+0000","last_active":"2024-09-10T22:38:40.597906+0000","last_peered":"2024-09-10T22:38:40.597906+0000","last_clean":"2024-09-10T22:38:40.597906+0000","last_became_active":"2024-09-10T22:38:39.603630+0000","last_became_peered":"2024-09-10T22:38:39.603630+0000","last_unstale":"2024-09-10T22:38:40.597906+0000","last_undegraded":"2024-09-10T22:38:40.597906+0000","last_fullsized":"2024-09-10T22:38:40.597906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:11.073421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607327+0000","last_change":"2024-09-10T22:38:40.602737+0000","last_active":"2024-09-10T22:38:40.607327+0000","last_peered":"2024-09-10T22:38:40.607327+0000","last_clean":"2024-09-10T22:38:40.607327+0000","last_became_active":"2024-09-10T22:38:40.602396+0000","last_became_peered":"2024-09-10T22:38:40.602396+0000","last_unstale":"2024-09-10T22:38:40.607327+0000","last_undegraded":"2024-09-10T22:38:40.607327+0000","last_fullsized":"2024-09-10T22:38:40.607327+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:02.952746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593568+0000","last_change":"2024-09-10T22:38:39.597398+0000","last_active":"2024-09-10T22:38:40.593568+0000","last_peered":"2024-09-10T22:38:40.593568+0000","last_clean":"2024-09-10T22:38:40.593568+0000","last_became_active":"2024-09-10T22:38:39.597124+0000","last_became_peered":"2024-09-10T22:38:39.597124+0000","last_unstale":"2024-09-10T22:38:40.593568+0000","last_undegraded":"2024-09-10T22:38:40.593568+0000","last_fullsized":"2024-09-10T22:38:40.593568+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:17:14.150710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.607913+0000","last_change":"2024-09-10T22:38:40.605128+0000","last_active":"2024-09-10T22:38:40.607913+0000","last_peered":"2024-09-10T22:38:40.607913+0000","last_clean":"2024-09-10T22:38:40.607913+0000","last_became_active":"2024-09-10T22:38:40.604859+0000","last_became_peered":"2024-09-10T22:38:40.604859+0000","last_unstale":"2024-09-10T22:38:40.607913+0000","last_undegraded":"2024-09-10T22:38:40.607913+0000","last_fullsized":"2024-09-10T22:38:40.607913+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:21.112573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.591895+0000","last_change":"2024-09-10T22:38:39.606428+0000","last_active":"2024-09-10T22:38:41.591895+0000","last_peered":"2024-09-10T22:38:41.591895+0000","last_clean":"2024-09-10T22:38:41.591895+0000","last_became_active":"2024-09-10T22:38:39.606030+0000","last_became_peered":"2024-09-10T22:38:39.606030+0000","last_unstale":"2024-09-10T22:38:41.591895+0000","last_undegraded":"2024-09-10T22:38:41.591895+0000","last_fullsized":"2024-09-10T22:38:41.591895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:05.772523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590641+0000","last_change":"2024-09-10T22:38:40.600176+0000","last_active":"2024-09-10T22:38:41.590641+0000","last_peered":"2024-09-10T22:38:41.590641+0000","last_clean":"2024-09-10T22:38:41.590641+0000","last_became_active":"2024-09-10T22:38:40.599888+0000","last_became_peered":"2024-09-10T22:38:40.599888+0000","last_unstale":"2024-09-10T22:38:41.590641+0000","last_undegraded":"2024-09-10T22:38:41.590641+0000","last_fullsized":"2024-09-10T22:38:41.590641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:39:05.125719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.597881+0000","last_change":"2024-09-10T22:38:39.601705+0000","last_active":"2024-09-10T22:38:40.597881+0000","last_peered":"2024-09-10T22:38:40.597881+0000","last_clean":"2024-09-10T22:38:40.597881+0000","last_became_active":"2024-09-10T22:38:39.601319+0000","last_became_peered":"2024-09-10T22:38:39.601319+0000","last_unstale":"2024-09-10T22:38:40.597881+0000","last_undegraded":"2024-09-10T22:38:40.597881+0000","last_fullsized":"2024-09-10T22:38:40.597881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:04.818960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590640+0000","last_change":"2024-09-10T22:38:40.602352+0000","last_active":"2024-09-10T22:38:41.590640+0000","last_peered":"2024-09-10T22:38:41.590640+0000","last_clean":"2024-09-10T22:38:41.590640+0000","last_became_active":"2024-09-10T22:38:40.601968+0000","last_became_peered":"2024-09-10T22:38:40.601968+0000","last_unstale":"2024-09-10T22:38:41.590640+0000","last_undegraded":"2024-09-10T22:38:41.590640+0000","last_fullsized":"2024-09-10T22:38:41.590640+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:44.324673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.602827+0000","last_change":"2024-09-10T22:38:39.611242+0000","last_active":"2024-09-10T22:38:40.602827+0000","last_peered":"2024-09-10T22:38:40.602827+0000","last_clean":"2024-09-10T22:38:40.602827+0000","last_became_active":"2024-09-10T22:38:39.610940+0000","last_became_peered":"2024-09-10T22:38:39.610940+0000","last_unstale":"2024-09-10T22:38:40.602827+0000","last_undegraded":"2024-09-10T22:38:40.602827+0000","last_fullsized":"2024-09-10T22:38:40.602827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:01.922287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.608082+0000","last_change":"2024-09-10T22:38:40.601507+0000","last_active":"2024-09-10T22:38:40.608082+0000","last_peered":"2024-09-10T22:38:40.608082+0000","last_clean":"2024-09-10T22:38:40.608082+0000","last_became_active":"2024-09-10T22:38:40.601224+0000","last_became_peered":"2024-09-10T22:38:40.601224+0000","last_unstale":"2024-09-10T22:38:40.608082+0000","last_undegraded":"2024-09-10T22:38:40.608082+0000","last_fullsized":"2024-09-10T22:38:40.608082+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:39.580141+0000","last_clean_scrub_stamp":"2024-09-10T22:38:39.580141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:54.564665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-10T22:38:41.590921+0000","last_change":"2024-09-10T22:38:39.606569+0000","last_active":"2024-09-10T22:38:41.590921+0000","last_peered":"2024-09-10T22:38:41.590921+0000","last_clean":"2024-09-10T22:38:41.590921+0000","last_became_active":"2024-09-10T22:38:39.606253+0000","last_became_peered":"2024-09-10T22:38:39.606253+0000","last_unstale":"2024-09-10T22:38:41.590921+0000","last_undegraded":"2024-09-10T22:38:41.590921+0000","last_fullsized":"2024-09-10T22:38:41.590921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:38:38.577833+0000","last_clean_scrub_stamp":"2024-09-10T22:38:38.577833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:42:03.427787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-10T22:38:40.593720+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:38:40.593720+0000","last_peered":"2024-09-10T22:38:40.593720+0000","last_clean":"2024-09-10T22:38:40.593720+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:38:40.593720+0000","last_undegraded":"2024-09-10T22:38:40.593720+0000","last_fullsized":"2024-09-10T22:38:40.593720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1130496,"data_stored":807171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":540672,"data_stored":217385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":536576,"data_stored":216783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":544768,"data_stored":218325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1130496,"data_stored":807115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":520192,"data_stored":209938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":209938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":520192,"data_stored":209938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:38:44.988 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:38:44.988 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 -- 172.21.15.66:0/2313418915 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe024074830 msgr2=0x7fe024072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2313418915 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe024074830 0x7fe024072810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe00c005fd0 tx=0x7fe00c02a5d0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 -- 172.21.15.66:0/2313418915 shutdown_connections 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2313418915 >> v2:172.21.15.94:3300/0 conn(0x7fe02415c340 0x7fe0240701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2313418915 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe024074830 0x7fe024072810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 -- 172.21.15.66:0/2313418915 >> 172.21.15.66:0/2313418915 conn(0x7fe0240b1950 msgr2=0x7fe0240b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 -- 172.21.15.66:0/2313418915 shutdown_connections 2024-09-10T22:38:45.139 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.137+0000 7fe0295e2640 1 -- 172.21.15.66:0/2313418915 wait complete. 2024-09-10T22:38:45.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.138+0000 7fe0295e2640 1 Processor -- start 2024-09-10T22:38:45.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe0295e2640 1 -- start start 2024-09-10T22:38:45.141 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe0295e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe024074450 0x7fe024159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe0295e2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe024074830 0x7fe0240a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe0295e2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe02415c340 0x7fe02415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe0295e2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe024070a90 con 0x7fe024074830 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe0295e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe024070910 con 0x7fe024074450 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe0295e2640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe02406e870 con 0x7fe02415c340 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe024074830 0x7fe0240a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe024074830 0x7fe0240a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48596/0 (socket says 172.21.15.66:48596) 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.139+0000 7fe01bfff640 1 -- 172.21.15.66:0/2902156500 learned_addr learned my addr 172.21.15.66:0/2902156500 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe0237fe640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe02415c340 0x7fe02415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe022ffd640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe024074450 0x7fe024159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01bfff640 1 -- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe02415c340 msgr2=0x7fe02415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01bfff640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe02415c340 0x7fe02415e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01bfff640 1 -- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe024074450 msgr2=0x7fe024159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.142 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01bfff640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe024074450 0x7fe024159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01bfff640 1 -- 172.21.15.66:0/2902156500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0241599b0 con 0x7fe024074830 2024-09-10T22:38:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01bfff640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe024074830 0x7fe0240a94d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0099f0 tx=0x7fe00c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01b7fe640 1 -- 172.21.15.66:0/2902156500 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c03f070 con 0x7fe024074830 2024-09-10T22:38:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01b7fe640 1 -- 172.21.15.66:0/2902156500 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe00c005600 con 0x7fe024074830 2024-09-10T22:38:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0241f2940 con 0x7fe024074830 2024-09-10T22:38:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe01b7fe640 1 -- 172.21.15.66:0/2902156500 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c035030 con 0x7fe024074830 2024-09-10T22:38:45.143 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.140+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0241f2f00 con 0x7fe024074830 2024-09-10T22:38:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.141+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe024072810 con 0x7fe024074830 2024-09-10T22:38:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.142+0000 7fe01b7fe640 1 -- 172.21.15.66:0/2902156500 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe00c004a00 con 0x7fe024074830 2024-09-10T22:38:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.142+0000 7fe01b7fe640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe004077fa0 0x7fe00407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.142+0000 7fe01b7fe640 1 -- 172.21.15.66:0/2902156500 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fe00c0bab00 con 0x7fe024074830 2024-09-10T22:38:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.143+0000 7fe022ffd640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe004077fa0 0x7fe00407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.144 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.143+0000 7fe022ffd640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe004077fa0 0x7fe00407a460 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe014007fd0 tx=0x7fe014009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:45.147 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.146+0000 7fe01b7fe640 1 -- 172.21.15.66:0/2902156500 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe00c082fb0 con 0x7fe024074830 2024-09-10T22:38:45.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.403+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe0240a9a10 con 0x7fe024074830 2024-09-10T22:38:45.406 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.405+0000 7fe01b7fe640 1 -- 172.21.15.66:0/2902156500 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fe00c0879e0 con 0x7fe024074830 2024-09-10T22:38:45.406 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T22:38:45.406 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQBcyeBmyY04GBAAcxW03/29zpbpxXzE+JrsFA== 2024-09-10T22:38:45.406 INFO:teuthology.orchestra.run.smithi066.stdout: caps mds = "allow" 2024-09-10T22:38:45.406 INFO:teuthology.orchestra.run.smithi066.stdout: caps mon = "allow r" 2024-09-10T22:38:45.406 INFO:teuthology.orchestra.run.smithi066.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T22:38:45.406 INFO:teuthology.orchestra.run.smithi066.stderr:updated caps for client.0 2024-09-10T22:38:45.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe004077fa0 msgr2=0x7fe00407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe004077fa0 0x7fe00407a460 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe014007fd0 tx=0x7fe014009040 comp rx=0 tx=0).stop 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe024074830 msgr2=0x7fe0240a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe024074830 0x7fe0240a94d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0099f0 tx=0x7fe00c003820 comp rx=0 tx=0).stop 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 shutdown_connections 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe004077fa0 0x7fe00407a460 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe02415c340 0x7fe02415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe024074830 0x7fe0240a94d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 --2- 172.21.15.66:0/2902156500 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe024074450 0x7fe024159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 >> 172.21.15.66:0/2902156500 conn(0x7fe0240b1950 msgr2=0x7fe0240cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 shutdown_connections 2024-09-10T22:38:45.409 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.407+0000 7fe0295e2640 1 -- 172.21.15.66:0/2902156500 wait complete. 2024-09-10T22:38:45.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 -- 172.21.15.66:0/866170181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 msgr2=0x7f503006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 --2- 172.21.15.66:0/866170181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006e870 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f50300742b0 tx=0x7f502002a5f0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 -- 172.21.15.66:0/866170181 shutdown_connections 2024-09-10T22:38:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 --2- 172.21.15.66:0/866170181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006e870 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 --2- 172.21.15.66:0/866170181 >> v2:172.21.15.94:3300/0 conn(0x7f5030070210 0x7f50300705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 -- 172.21.15.66:0/866170181 >> 172.21.15.66:0/866170181 conn(0x7f50300b1950 msgr2=0x7f50300b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 -- 172.21.15.66:0/866170181 shutdown_connections 2024-09-10T22:38:45.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.523+0000 7f50383f3640 1 -- 172.21.15.66:0/866170181 wait complete. 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.524+0000 7f50383f3640 1 Processor -- start 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.524+0000 7f50383f3640 1 -- start start 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f50383f3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5030070210 0x7f50300c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f50383f3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f50383f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f503015e800 0x7f50300c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f50383f3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f50300cb4c0 con 0x7f5030073040 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f50383f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5030152170 con 0x7f503015e800 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f50383f3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f50300cb640 con 0x7f5030070210 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.527 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48620/0 (socket says 172.21.15.66:48620) 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 -- 172.21.15.66:0/1862654430 learned_addr learned my addr 172.21.15.66:0/1862654430 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503618f640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5030070210 0x7f50300c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f5036990640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f503015e800 0x7f50300c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 -- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5030070210 msgr2=0x7f50300c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5030070210 0x7f50300c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 -- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f503015e800 msgr2=0x7f50300c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f503015e800 0x7f50300c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.525+0000 7f503598e640 1 -- 172.21.15.66:0/1862654430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50301f4eb0 con 0x7f5030073040 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.526+0000 7f503598e640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006f370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5020005fd0 tx=0x7f50200038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.526+0000 7f5026ffd640 1 -- 172.21.15.66:0/1862654430 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502003f070 con 0x7f5030073040 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.526+0000 7f5026ffd640 1 -- 172.21.15.66:0/1862654430 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50200047a0 con 0x7f5030073040 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.526+0000 7f5026ffd640 1 -- 172.21.15.66:0/1862654430 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020035030 con 0x7f5030073040 2024-09-10T22:38:45.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.526+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50301f5190 con 0x7f5030073040 2024-09-10T22:38:45.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.526+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5030152c50 con 0x7f5030073040 2024-09-10T22:38:45.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.527+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503015f370 con 0x7f5030073040 2024-09-10T22:38:45.529 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.527+0000 7f5026ffd640 1 -- 172.21.15.66:0/1862654430 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5020004bf0 con 0x7f5030073040 2024-09-10T22:38:45.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.528+0000 7f5026ffd640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5014077fa0 0x7f501407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:45.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.528+0000 7f5026ffd640 1 -- 172.21.15.66:0/1862654430 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f50200bb0c0 con 0x7f5030073040 2024-09-10T22:38:45.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.528+0000 7f503618f640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5014077fa0 0x7f501407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:45.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.529+0000 7f503618f640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5014077fa0 0x7f501407a460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f502c009bd0 tx=0x7f502c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:45.533 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.532+0000 7f5026ffd640 1 -- 172.21.15.66:0/1862654430 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5020083600 con 0x7f5030073040 2024-09-10T22:38:45.756 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.755+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5030074ad0 con 0x7f5030073040 2024-09-10T22:38:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.756+0000 7f5026ffd640 1 -- 172.21.15.66:0/1862654430 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5223 (secure 0 0 0) 0x7f5020088030 con 0x7f5030073040 2024-09-10T22:38:45.757 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:45.757 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":32,"btime":"2024-09-10T22:38:43:613077+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3548861128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3548861128},{"type":"v1","addr":"172.21.15.94:6837","nonce":3548861128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1690198522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1690198522},{"type":"v1","addr":"172.21.15.94:6836","nonce":1690198522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1833881771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1833881771},{"type":"v1","addr":"172.21.15.66:6834","nonce":1833881771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:42.976687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3999038135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3999038135},{"type":"v1","addr":"172.21.15.66:6835","nonce":3999038135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5304,"qdb_cluster":[5304]},"id":3}]} 2024-09-10T22:38:45.757 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 32 2024-09-10T22:38:45.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5014077fa0 msgr2=0x7f501407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5014077fa0 0x7f501407a460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f502c009bd0 tx=0x7f502c005f90 comp rx=0 tx=0).stop 2024-09-10T22:38:45.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 msgr2=0x7f503006f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:45.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006f370 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5020005fd0 tx=0x7f50200038a0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.760 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 shutdown_connections 2024-09-10T22:38:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5014077fa0 0x7f501407a460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f503015e800 0x7f50300c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5030073040 0x7f503006f370 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 --2- 172.21.15.66:0/1862654430 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5030070210 0x7f50300c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 >> 172.21.15.66:0/1862654430 conn(0x7f50300b1950 msgr2=0x7f50300b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:45.761 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 shutdown_connections 2024-09-10T22:38:45.762 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:45.758+0000 7f50383f3640 1 -- 172.21.15.66:0/1862654430 wait complete. 2024-09-10T22:38:46.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:38:39.959431+0000', 'modified': '2024-09-10T22:38:42.976687+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5304': {'gid': 5304, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/3999038135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 3999038135}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 3999038135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5304, 'qdb_cluster': [5304]} 2024-09-10T22:38:46.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:38:46.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:38:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 -- 172.21.15.66:0/2668590981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 msgr2=0x7f4118162580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 --2- 172.21.15.66:0/2668590981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 0x7f4118162580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4108009830 tx=0x7f410802a480 comp rx=0 tx=0).stop 2024-09-10T22:38:46.282 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 -- 172.21.15.66:0/2668590981 shutdown_connections 2024-09-10T22:38:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 --2- 172.21.15.66:0/2668590981 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cea20 0x7f41180ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 --2- 172.21.15.66:0/2668590981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 0x7f4118162580 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 -- 172.21.15.66:0/2668590981 >> 172.21.15.66:0/2668590981 conn(0x7f41180d3f10 msgr2=0x7f41180d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 -- 172.21.15.66:0/2668590981 shutdown_connections 2024-09-10T22:38:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.281+0000 7f411c99f640 1 -- 172.21.15.66:0/2668590981 wait complete. 2024-09-10T22:38:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.282+0000 7f411c99f640 1 Processor -- start 2024-09-10T22:38:46.283 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.282+0000 7f411c99f640 1 -- start start 2024-09-10T22:38:46.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f411c99f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cea20 0x7f41180cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f411c99f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41180d1ae0 0x7f4118155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f411c99f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 0x7f41180cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.291 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f411c99f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41180cf3f0 con 0x7f41180d1ae0 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f411c99f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41180d2060 con 0x7f41180d24f0 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f411c99f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f41180cf570 con 0x7f41180cea20 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f4116ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41180d1ae0 0x7f4118155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f4116ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41180d1ae0 0x7f4118155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48628/0 (socket says 172.21.15.66:48628) 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f4116ffd640 1 -- 172.21.15.66:0/1489508090 learned_addr learned my addr 172.21.15.66:0/1489508090 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f41177fe640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cea20 0x7f41180cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.283+0000 7f4117fff640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 0x7f41180cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f4116ffd640 1 -- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cea20 msgr2=0x7f41180cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f4116ffd640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cea20 0x7f41180cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f4116ffd640 1 -- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 msgr2=0x7f41180cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f4116ffd640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 0x7f41180cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f4116ffd640 1 -- 172.21.15.66:0/1489508090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41180b1920 con 0x7f41180d1ae0 2024-09-10T22:38:46.292 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f4116ffd640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41180d1ae0 0x7f4118155270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c480 tx=0x7f410c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f40fffff640 1 -- 172.21.15.66:0/1489508090 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c016020 con 0x7f41180d1ae0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f40fffff640 1 -- 172.21.15.66:0/1489508090 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f410c012920 con 0x7f41180d1ae0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41180b1ba0 con 0x7f41180d1ae0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f40fffff640 1 -- 172.21.15.66:0/1489508090 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c012d70 con 0x7f41180d1ae0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.284+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41180b2160 con 0x7f41180d1ae0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.286+0000 7f40fffff640 1 -- 172.21.15.66:0/1489508090 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f410c01b400 con 0x7f41180d1ae0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.286+0000 7f40fffff640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4104077fa0 0x7f410407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.286+0000 7f40fffff640 1 -- 172.21.15.66:0/1489508090 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f410c099f00 con 0x7f41180d1ae0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.287+0000 7f41177fe640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4104077fa0 0x7f410407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.287+0000 7f41209ce740 1 Processor -- start 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.287+0000 7f41177fe640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4104077fa0 0x7f410407a460 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4108034000 tx=0x7f4108004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.287+0000 7f41209ce740 1 -- start start 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41209ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d130cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d1793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d12e7c0 0x557d2d12ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41209ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557d2d0a5820 con 0x557d2d159380 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557d2d094c50 con 0x557d2d178fc0 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x557d2d098a60 con 0x557d2d12e7c0 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d130cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d130cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48634/0 (socket says 172.21.15.66:48634) 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 -- 172.21.15.66:0/2303132768 learned_addr learned my addr 172.21.15.66:0/2303132768 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f4116ffd640 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d1793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f4117fff640 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d12e7c0 0x557d2d12ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 -- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d12e7c0 msgr2=0x557d2d12ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d12e7c0 0x557d2d12ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 -- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 msgr2=0x557d2d1793c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.294 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d1793c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.288+0000 7f41177fe640 1 -- 172.21.15.66:0/2303132768 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d2cdf6850 con 0x557d2d159380 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.289+0000 7f41177fe640 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d130cf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4108009780 tx=0x7f410803e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9ed8f018ee8987 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.289+0000 7f40fd7fa640 1 -- 172.21.15.66:0/2303132768 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108003d40 con 0x557d2d159380 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.289+0000 7f40fd7fa640 1 -- 172.21.15.66:0/2303132768 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f410804b070 con 0x557d2d159380 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.289+0000 7f40fd7fa640 1 -- 172.21.15.66:0/2303132768 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080506d0 con 0x557d2d159380 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.289+0000 7f41209ce740 1 -- 172.21.15.66:0/2303132768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 msgr2=0x557d2d130cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.289+0000 7f41209ce740 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d130cf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4108009780 tx=0x7f410803e070 comp rx=0 tx=0).stop 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 -- 172.21.15.66:0/2303132768 shutdown_connections 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d12e7c0 0x557d2d12ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d1793c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 --2- 172.21.15.66:0/2303132768 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d130cf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 -- 172.21.15.66:0/2303132768 >> 172.21.15.66:0/2303132768 conn(0x557d2d12f870 msgr2=0x557d2d133290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 -- 172.21.15.66:0/2303132768 shutdown_connections 2024-09-10T22:38:46.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 -- 172.21.15.66:0/2303132768 wait complete. 2024-09-10T22:38:46.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 -- start start 2024-09-10T22:38:46.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.290+0000 7f41209ce740 1 Processor -- start 2024-09-10T22:38:46.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 0x557d2d154980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 0x557d2d150520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f41209ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d150a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f41209ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557d2ce05620 con 0x557d2d154ec0 2024-09-10T22:38:46.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557d2d03a5f0 con 0x557d2d159380 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x557d2d12da60 con 0x557d2d178fc0 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f41209ce740 10 client.0 _ll_register_callbacks cb 0x557d2d03e0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48644/0 (socket says 172.21.15.66:48644) 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 -- 172.21.15.66:0/43880107 learned_addr learned my addr 172.21.15.66:0/43880107 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 -- 172.21.15.66:0/43880107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 msgr2=0x557d2d150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 0x557d2d150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 -- 172.21.15.66:0/43880107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 msgr2=0x557d2d154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 0x557d2d154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.291+0000 7f4117fff640 1 -- 172.21.15.66:0/43880107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d2ce6d6b0 con 0x557d2d154ec0 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f4117fff640 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d150a60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4110011d40 tx=0x7f411000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430c6757ba78f12a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f40f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f40f67fc640 1 -- 172.21.15.66:0/43880107 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411000fe60 con 0x557d2d154ec0 2024-09-10T22:38:46.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f40f67fc640 1 -- 172.21.15.66:0/43880107 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f4110003970 con 0x557d2d154ec0 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f410804b070 con 0x557d2d154ec0 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f40f67fc640 1 -- 172.21.15.66:0/43880107 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4110024050 con 0x557d2d154ec0 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 10 client.5418 fetch_fsmap learned FSMap version 32 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x557d2d152760 con 0x557d2d154ec0 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f40f67fc640 1 -- 172.21.15.66:0/43880107 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f410804b070 con 0x557d2d154ec0 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f40f67fc640 1 -- 172.21.15.66:0/43880107 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 3502+0+0 (secure 0 0 0) 0x7f4110027020 con 0x557d2d154ec0 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 10 client.5418 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 10 client.5418 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 20 client.5418 populate_metadata read hostname 'smithi066' 2024-09-10T22:38:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d1530e0 0x557d2d187ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 4 client.5418 mds_command: new command op to 5304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 --> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d2d0a7770 con 0x557d2d1530e0 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.292+0000 7f41177fe640 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d1530e0 0x557d2d187ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f41177fe640 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d1530e0 0x557d2d187ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cc9f8958c9e1a5 server_cookie=2122608c2b5831c1 in_seq=0 out_seq=0 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f40f67fc640 10 client.5418 ms_handle_connect on v2:172.21.15.66:6833/3999038135 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f40f67fc640 1 -- 172.21.15.66:0/43880107 <== mds.0 v2:172.21.15.66:6833/3999038135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d2d0a7770 con 0x557d2d1530e0 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f40f67fc640 10 client.5418 handle_command_reply: tid=0 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f41209ce740 1 client.5418 shutdown 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 msgr2=0x557d2d150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f41209ce740 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d150a60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4110011d40 tx=0x7f411000f790 comp rx=0 tx=0).stop 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 shutdown_connections 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.293+0000 7f41209ce740 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d1530e0 0x557d2d187ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f41209ce740 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d150a60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f41209ce740 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 0x557d2d150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f40f67fc640 0 client.5418 ms_handle_reset on v2:172.21.15.66:6833/3999038135 2024-09-10T22:38:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f41209ce740 1 --2- 172.21.15.66:0/43880107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 0x557d2d154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 >> 172.21.15.66:0/43880107 conn(0x557d2d12e7c0 msgr2=0x557d2d159760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 shutdown_connections 2024-09-10T22:38:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f41209ce740 1 -- 172.21.15.66:0/43880107 wait complete. 2024-09-10T22:38:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.294+0000 7f41209ce740 20 client.5418 trim_cache size 0 max 16384 2024-09-10T22:38:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.312+0000 7f41209ce740 1 Processor -- start 2024-09-10T22:38:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.312+0000 7f41209ce740 1 -- start start 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41209ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d159950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 0x557d2d157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 0x557d2d158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41209ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f41182001f0 con 0x557d2d154ec0 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41181f8c60 con 0x557d2d159380 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f411815da00 con 0x557d2d178fc0 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48656/0 (socket says 172.21.15.66:48656) 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 -- 172.21.15.66:0/3981554085 learned_addr learned my addr 172.21.15.66:0/3981554085 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 -- 172.21.15.66:0/3981554085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 msgr2=0x557d2d158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 --2- 172.21.15.66:0/3981554085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 0x557d2d158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 -- 172.21.15.66:0/3981554085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 msgr2=0x557d2d157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 --2- 172.21.15.66:0/3981554085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 0x557d2d157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 -- 172.21.15.66:0/3981554085 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d2d15a550 con 0x557d2d154ec0 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f41177fe640 1 --2- 172.21.15.66:0/3981554085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d159950 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f410802a4c0 tx=0x7f410803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00f790aaf6d1480 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f40fcff9640 1 -- 172.21.15.66:0/3981554085 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410805c020 con 0x557d2d154ec0 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f40fcff9640 1 -- 172.21.15.66:0/3981554085 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f410804b050 con 0x557d2d154ec0 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.313+0000 7f40fcff9640 1 -- 172.21.15.66:0/3981554085 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108050690 con 0x557d2d154ec0 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 -- 172.21.15.66:0/3981554085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 msgr2=0x557d2d159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 --2- 172.21.15.66:0/3981554085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d159950 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f410802a4c0 tx=0x7f410803e000 comp rx=0 tx=0).stop 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 -- 172.21.15.66:0/3981554085 shutdown_connections 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 --2- 172.21.15.66:0/3981554085 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d178fc0 0x557d2d158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 --2- 172.21.15.66:0/3981554085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d159380 0x557d2d157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 --2- 172.21.15.66:0/3981554085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d154ec0 0x557d2d159950 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 -- 172.21.15.66:0/3981554085 >> 172.21.15.66:0/3981554085 conn(0x557d2d12e7c0 msgr2=0x557d2d12eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 -- 172.21.15.66:0/3981554085 shutdown_connections 2024-09-10T22:38:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.314+0000 7f41209ce740 1 -- 172.21.15.66:0/3981554085 wait complete. 2024-09-10T22:38:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.315+0000 7f41209ce740 1 -- start start 2024-09-10T22:38:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.315+0000 7f41209ce740 1 Processor -- start 2024-09-10T22:38:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d154ec0 0x557d2d12d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d132960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d132ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x557d2d152f60 con 0x557d2d159380 2024-09-10T22:38:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557d2d15aed0 con 0x557d2d178fc0 2024-09-10T22:38:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x557d2d1535d0 con 0x557d2d154ec0 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 10 client.0 _ll_register_callbacks cb 0x7f4110009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d132960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d132960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48660/0 (socket says 172.21.15.66:48660) 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 -- 172.21.15.66:0/1455706165 learned_addr learned my addr 172.21.15.66:0/1455706165 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41177fe640 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d154ec0 0x557d2d12d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4117fff640 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d132ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 -- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d154ec0 msgr2=0x557d2d12d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d154ec0 0x557d2d12d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 -- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 msgr2=0x557d2d132ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d132ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 -- 172.21.15.66:0/1455706165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d2d179780 con 0x557d2d159380 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f4116ffd640 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d132960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f410c098420 tx=0x7f410c05ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8795347f0b008054 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f40f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f40f67fc640 1 -- 172.21.15.66:0/1455706165 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c0623a0 con 0x557d2d159380 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f40f67fc640 1 -- 172.21.15.66:0/1455706165 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f410c061600 con 0x557d2d159380 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f410804b050 con 0x557d2d159380 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f40f67fc640 1 -- 172.21.15.66:0/1455706165 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f410c066dd0 con 0x557d2d159380 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 10 client.5424 fetch_fsmap learned FSMap version 32 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.316+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x557d2d201610 con 0x557d2d159380 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f40f67fc640 1 -- 172.21.15.66:0/1455706165 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f410804b050 con 0x557d2d159380 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f40f67fc640 1 -- 172.21.15.66:0/1455706165 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 3502+0+0 (secure 0 0 0) 0x7f410c05dcf0 con 0x557d2d159380 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 10 client.5424 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 10 client.5424 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 20 client.5424 populate_metadata read hostname 'smithi066' 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d205250 0x557d2d157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 4 client.5424 mds_command: new command op to 5304 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 --> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] -- command(tid 0: {"prefix": "status"}) -- 0x557d2d0a3220 con 0x557d2d205250 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41177fe640 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d205250 0x557d2d157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41177fe640 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d205250 0x557d2d157d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6317e0c9dcf1d1e server_cookie=2cd6e7d1eec029ff in_seq=0 out_seq=0 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f40f67fc640 10 client.5424 ms_handle_connect on v2:172.21.15.66:6833/3999038135 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f40f67fc640 1 -- 172.21.15.66:0/1455706165 <== mds.0 v2:172.21.15.66:6833/3999038135 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x557d2d0a3220 con 0x557d2d205250 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f40f67fc640 10 client.5424 handle_command_reply: tid=0 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 1 client.5424 shutdown 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 msgr2=0x557d2d132960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d132960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f410c098420 tx=0x7f410c05ea80 comp rx=0 tx=0).stop 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 shutdown_connections 2024-09-10T22:38:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.317+0000 7f41209ce740 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x557d2d205250 0x557d2d157d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f41209ce740 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d2d178fc0 0x557d2d132ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f40f67fc640 0 client.5424 ms_handle_reset on v2:172.21.15.66:6833/3999038135 2024-09-10T22:38:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f41209ce740 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557d2d159380 0x557d2d132960 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f41209ce740 1 --2- 172.21.15.66:0/1455706165 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x557d2d154ec0 0x557d2d12d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 >> 172.21.15.66:0/1455706165 conn(0x557d2d12e7c0 msgr2=0x557d2d186f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 shutdown_connections 2024-09-10T22:38:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f41209ce740 1 -- 172.21.15.66:0/1455706165 wait complete. 2024-09-10T22:38:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f41209ce740 20 client.5424 trim_cache size 0 max 16384 2024-09-10T22:38:46.327 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:38:46.327 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "7ef137cd-a1ce-4819-b27d-b71268612f85", 2024-09-10T22:38:46.327 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:38:46.328 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5304, 2024-09-10T22:38:46.328 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:38:46.328 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:38:46.328 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:38:46.328 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 5.7312879060000004, 2024-09-10T22:38:46.329 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 32, 2024-09-10T22:38:46.329 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 47, 2024-09-10T22:38:46.329 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 47, 2024-09-10T22:38:46.329 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 10.399817336 2024-09-10T22:38:46.329 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:38:46.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4104077fa0 msgr2=0x7f410407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.331 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4104077fa0 0x7f410407a460 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4108034000 tx=0x7f4108004730 comp rx=0 tx=0).stop 2024-09-10T22:38:46.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41180d1ae0 msgr2=0x7f4118155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41180d1ae0 0x7f4118155270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c480 tx=0x7f410c00c940 comp rx=0 tx=0).stop 2024-09-10T22:38:46.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 shutdown_connections 2024-09-10T22:38:46.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f4104077fa0 0x7f410407a460 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41180d24f0 0x7f41180cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.332 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f41180d1ae0 0x7f4118155270 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 --2- 172.21.15.66:0/1489508090 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f41180cea20 0x7f41180cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 >> 172.21.15.66:0/1489508090 conn(0x7f41180d3f10 msgr2=0x7f411806d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 shutdown_connections 2024-09-10T22:38:46.333 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.318+0000 7f411c99f640 1 -- 172.21.15.66:0/1489508090 wait complete. 2024-09-10T22:38:46.333 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T22:38:46.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 -- 172.21.15.66:0/3959719344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360073040 msgr2=0x7f636006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3959719344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360073040 0x7f636006e870 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6354005fd0 tx=0x7f635402a5d0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 -- 172.21.15.66:0/3959719344 shutdown_connections 2024-09-10T22:38:46.439 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3959719344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360073040 0x7f636006e870 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3959719344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6360070210 0x7f63600705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 -- 172.21.15.66:0/3959719344 >> 172.21.15.66:0/3959719344 conn(0x7f63600b1950 msgr2=0x7f63600b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 -- 172.21.15.66:0/3959719344 shutdown_connections 2024-09-10T22:38:46.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.438+0000 7f6365e7f640 1 -- 172.21.15.66:0/3959719344 wait complete. 2024-09-10T22:38:46.440 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.439+0000 7f6365e7f640 1 Processor -- start 2024-09-10T22:38:46.441 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.439+0000 7f6365e7f640 1 -- start start 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f6365e7f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360070210 0x7f6360146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f6365e7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6360073040 0x7f6360153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f6365e7f640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f636015e800 0x7f63601543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f6365e7f640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f636015f3a0 con 0x7f6360070210 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f6365e7f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6360152170 con 0x7f6360073040 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f6365e7f640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f636015f520 con 0x7f636015e800 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360070210 0x7f6360146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360070210 0x7f6360146140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48684/0 (socket says 172.21.15.66:48684) 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635f7fe640 1 -- 172.21.15.66:0/3560016247 learned_addr learned my addr 172.21.15.66:0/3560016247 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635effd640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6360073040 0x7f6360153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635ffff640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f636015e800 0x7f63601543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635f7fe640 1 -- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f636015e800 msgr2=0x7f63601543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.443 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635f7fe640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f636015e800 0x7f63601543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.440+0000 7f635f7fe640 1 -- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6360073040 msgr2=0x7f6360153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f635f7fe640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6360073040 0x7f6360153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f635f7fe640 1 -- 172.21.15.66:0/3560016247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6360146680 con 0x7f6360070210 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f635f7fe640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360070210 0x7f6360146140 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f635000e9f0 tx=0x7f635000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f6347fff640 1 -- 172.21.15.66:0/3560016247 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635000cd80 con 0x7f6360070210 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f6347fff640 1 -- 172.21.15.66:0/3560016247 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6350010430 con 0x7f6360070210 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f6347fff640 1 -- 172.21.15.66:0/3560016247 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350010840 con 0x7f6360070210 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63601f4e00 con 0x7f6360070210 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.441+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63601f53c0 con 0x7f6360070210 2024-09-10T22:38:46.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.442+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63600cb5a0 con 0x7f6360070210 2024-09-10T22:38:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.442+0000 7f6347fff640 1 -- 172.21.15.66:0/3560016247 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f63500109e0 con 0x7f6360070210 2024-09-10T22:38:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.443+0000 7f6347fff640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f634c077f70 0x7f634c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.443+0000 7f6347fff640 1 -- 172.21.15.66:0/3560016247 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f635009a830 con 0x7f6360070210 2024-09-10T22:38:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.443+0000 7f635effd640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f634c077f70 0x7f634c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:46.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.444+0000 7f635effd640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f634c077f70 0x7f634c07a430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6354033000 tx=0x7f6354004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:46.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.447+0000 7f6347fff640 1 -- 172.21.15.66:0/3560016247 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6350062da0 con 0x7f6360070210 2024-09-10T22:38:46.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.669+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6360074ad0 con 0x7f6360070210 2024-09-10T22:38:46.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.669+0000 7f6347fff640 1 -- 172.21.15.66:0/3560016247 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+73 (secure 0 0 0) 0x7f63500677d0 con 0x7f6360070210 2024-09-10T22:38:46.670 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f634c077f70 msgr2=0x7f634c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f634c077f70 0x7f634c07a430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6354033000 tx=0x7f6354004730 comp rx=0 tx=0).stop 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360070210 msgr2=0x7f6360146140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360070210 0x7f6360146140 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f635000e9f0 tx=0x7f635000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 shutdown_connections 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f634c077f70 0x7f634c07a430 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f636015e800 0x7f63601543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6360073040 0x7f6360153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 --2- 172.21.15.66:0/3560016247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6360070210 0x7f6360146140 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:46.673 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.671+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 >> 172.21.15.66:0/3560016247 conn(0x7f63600b1950 msgr2=0x7f636006f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:46.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.672+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 shutdown_connections 2024-09-10T22:38:46.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46.672+0000 7f6365e7f640 1 -- 172.21.15.66:0/3560016247 wait complete. 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:38:46.684 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:38:46.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:46.685 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2024-09-10T22:38:46.701 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:38:46.702 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 42571sec preferred_lft 42571sec 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link noprefixroute 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:f5:87:39:84:80 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::f82d:d6ff:fec7:dfcb/64 scope link 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:38:46.703 INFO:teuthology.orchestra.run.smithi066.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether f2:19:d9:d3:e1:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::f019:d9ff:fed3:e1f4/64 scope link 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run.smithi066.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 5e:f5:87:39:84:80 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::5cf5:87ff:fe39:8480/64 scope link 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:38:46.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:46.704 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:38:46.766 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T22:38:46.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:46.768 DEBUG:teuthology.orchestra.run.smithi066:> route 2024-09-10T22:38:46.825 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2024-09-10T22:38:46.825 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:38:46.825 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:38:46.825 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:38:46.825 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:38:46.826 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:38:46.826 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2024-09-10T22:38:46.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:38:46.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:38:46.826 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:38:46.826 DEBUG:teuthology.orchestra.run.smithi066:> ') 2024-09-10T22:38:46.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:38:46.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:46.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:46.955 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:38:46.971 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:38:46.971 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:38:46.971 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:38:46.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:46.972 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2024-09-10T22:38:47.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:38:47.037 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:38:47.093 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:38:47.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:38:47.093 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:38:47.148 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T22:38:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:47.149 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:38:47.233 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:38:47.234 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:38:47.241 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T22:38:47.241 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T22:38:47.241 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.94:3300,172.21.15.94:3301" 2024-09-10T22:38:47.242 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with new device syntax: 0@7ef137cd-a1ce-4819-b27d-b71268612f85.cephfs=/ 2024-09-10T22:38:47.242 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.66:3300/172.21.15.94:3300/172.21.15.94:3301" will pass to kernel 2024-09-10T22:38:47.242 INFO:teuthology.orchestra.run.smithi066.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T22:38:47.242 INFO:teuthology.orchestra.run.smithi066.stdout: You just mounted a file system that supports labels which does not 2024-09-10T22:38:47.242 INFO:teuthology.orchestra.run.smithi066.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T22:38:47.242 INFO:teuthology.orchestra.run.smithi066.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T22:38:47.242 INFO:teuthology.orchestra.run.smithi066.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T22:38:47.244 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T22:38:47.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:38:47.244 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:38:47.270 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T22:38:47.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:38:47.367 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:38:47.372 INFO:teuthology.orchestra.run.smithi066.stdout:client4985 2024-09-10T22:38:47.373 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:38:47.373 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:38:47.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:38:47.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:38:47.475 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-10T22:38:47.475 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-09-10T22:38:47.476 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-09-10T22:38:47.477 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-09-10T22:38:47.478 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on skipped message = False 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-09-10T22:38:47.479 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.480 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:38:47.481 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:47.482 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 30 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:47.483 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-09-10T22:38:47.484 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T22:38:47.485 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi066.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.486 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-10T22:38:47.487 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.488 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on skipped message = False 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-10T22:38:47.489 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-10T22:38:47.490 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-10T22:38:47.491 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:38:47.492 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 30 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.493 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:38:47.494 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-10T22:38:47.495 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2024-09-10T22:38:47.496 INFO:teuthology.orchestra.run.smithi094.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-09-10T22:38:47.498 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T22:38:47.499 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T22:38:47.499 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:38:47.499 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:47.499 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T22:38:47.499 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T22:38:47.499 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:38:47.499 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:47.499 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T22:38:47.500 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T22:38:47.500 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:38:47.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:47.500 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T22:38:47.500 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T22:38:47.500 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:38:47.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:38:47.513 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:38:47.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T22:38:47.513 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:38:47.514 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T22:38:47.550 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:38:47.550 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T22:38:47.551 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:38:47.551 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T22:38:47.556 INFO:tasks.ceph.mds.b:Started 2024-09-10T22:38:47.556 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:47.557 INFO:tasks.ceph.mds.d:Started 2024-09-10T22:38:47.557 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:47.593 INFO:tasks.ceph.mds.a:Started 2024-09-10T22:38:47.594 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:47.594 INFO:tasks.ceph.mds.c:Started 2024-09-10T22:38:47.595 DEBUG:teuthology.parallel:result is None 2024-09-10T22:38:47.595 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:38:47.637 INFO:tasks.ceph.mds.b.smithi094.stdout:starting mds.b at 2024-09-10T22:38:47.639 INFO:tasks.ceph.mds.d.smithi094.stdout:starting mds.d at 2024-09-10T22:38:47.674 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:38:47.674 INFO:tasks.ceph.mds.a.smithi066.stdout:starting mds.a at 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.706+0000 7fb16b9a2640 1 -- 172.21.15.66:0/626977555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1640d4000 msgr2=0x7fb1640d43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.706+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/626977555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1640d4000 0x7fb1640d43e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb160009830 tx=0x7fb16002a510 comp rx=0 tx=0).stop 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.707+0000 7fb16b9a2640 1 -- 172.21.15.66:0/626977555 shutdown_connections 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.707+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/626977555 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1640b1fb0 0x7fb1640b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.707+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/626977555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1640d4000 0x7fb1640d43e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.707+0000 7fb16b9a2640 1 -- 172.21.15.66:0/626977555 >> 172.21.15.66:0/626977555 conn(0x7fb164074c20 msgr2=0x7fb16409d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.707+0000 7fb16b9a2640 1 -- 172.21.15.66:0/626977555 shutdown_connections 2024-09-10T22:38:47.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.707+0000 7fb16b9a2640 1 -- 172.21.15.66:0/626977555 wait complete. 2024-09-10T22:38:47.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.708+0000 7fb16b9a2640 1 Processor -- start 2024-09-10T22:38:47.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.708+0000 7fb16b9a2640 1 -- start start 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16b9a2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1640b1fb0 0x7fb1641f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16b9a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1640d4000 0x7fb16415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16b9a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16415e2b0 0x7fb1640b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16b9a2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640b2b00 con 0x7fb1640d4000 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16b9a2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640b2980 con 0x7fb16415e2b0 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16b9a2640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb1640b2c80 con 0x7fb1640b1fb0 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1640d4000 0x7fb16415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1640d4000 0x7fb16415abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48720/0 (socket says 172.21.15.66:48720) 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 -- 172.21.15.66:0/1357123718 learned_addr learned my addr 172.21.15.66:0/1357123718 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16973e640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1640b1fb0 0x7fb1641f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb169f3f640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16415e2b0 0x7fb1640b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:47.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 -- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1640b1fb0 msgr2=0x7fb1641f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1640b1fb0 0x7fb1641f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 -- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16415e2b0 msgr2=0x7fb1640b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16415e2b0 0x7fb1640b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 -- 172.21.15.66:0/1357123718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16415f9e0 con 0x7fb1640d4000 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb168f3d640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1640d4000 0x7fb16415abc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c430 tx=0x7fb15400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb159ffb640 1 -- 172.21.15.66:0/1357123718 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154017070 con 0x7fb1640d4000 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16415d4f0 con 0x7fb1640d4000 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb159ffb640 1 -- 172.21.15.66:0/1357123718 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1540053c0 con 0x7fb1640d4000 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.709+0000 7fb159ffb640 1 -- 172.21.15.66:0/1357123718 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154003850 con 0x7fb1640d4000 2024-09-10T22:38:47.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.710+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16415c730 con 0x7fb1640d4000 2024-09-10T22:38:47.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.711+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1640b3210 con 0x7fb1640d4000 2024-09-10T22:38:47.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.711+0000 7fb159ffb640 1 -- 172.21.15.66:0/1357123718 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb1540039f0 con 0x7fb1640d4000 2024-09-10T22:38:47.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.712+0000 7fb159ffb640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb148077f70 0x7fb14807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:47.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.712+0000 7fb159ffb640 1 -- 172.21.15.66:0/1357123718 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fb154099410 con 0x7fb1640d4000 2024-09-10T22:38:47.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.712+0000 7fb16973e640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb148077f70 0x7fb14807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:47.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.712+0000 7fb16973e640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb148077f70 0x7fb14807a430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb160034000 tx=0x7fb160004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:47.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.716+0000 7fb159ffb640 1 -- 172.21.15.66:0/1357123718 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb154061840 con 0x7fb1640d4000 2024-09-10T22:38:47.937 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.936+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb16415fbb0 con 0x7fb1640d4000 2024-09-10T22:38:47.938 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.937+0000 7fb159ffb640 1 -- 172.21.15.66:0/1357123718 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5223 (secure 0 0 0) 0x7fb154066270 con 0x7fb1640d4000 2024-09-10T22:38:47.938 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:38:47.938 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":32,"btime":"2024-09-10T22:38:43:613077+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/3548861128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6835","nonce":3548861128},{"type":"v1","addr":"172.21.15.94:6837","nonce":3548861128}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5167,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6836/1690198522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1690198522},{"type":"v1","addr":"172.21.15.94:6836","nonce":1690198522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1833881771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1833881771},{"type":"v1","addr":"172.21.15.66:6834","nonce":1833881771}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:42.976687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.66:6835/3999038135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":3999038135},{"type":"v1","addr":"172.21.15.66:6835","nonce":3999038135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5304,"qdb_cluster":[5304]},"id":3}]} 2024-09-10T22:38:47.938 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 32 2024-09-10T22:38:47.940 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb148077f70 msgr2=0x7fb14807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb148077f70 0x7fb14807a430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb160034000 tx=0x7fb160004730 comp rx=0 tx=0).stop 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1640d4000 msgr2=0x7fb16415abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1640d4000 0x7fb16415abc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c430 tx=0x7fb15400c8f0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 shutdown_connections 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb148077f70 0x7fb14807a430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16415e2b0 0x7fb1640b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1640d4000 0x7fb16415abc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 --2- 172.21.15.66:0/1357123718 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1640b1fb0 0x7fb1641f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 >> 172.21.15.66:0/1357123718 conn(0x7fb164074c20 msgr2=0x7fb1640b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 shutdown_connections 2024-09-10T22:38:47.941 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:47.939+0000 7fb16b9a2640 1 -- 172.21.15.66:0/1357123718 wait complete. 2024-09-10T22:38:48.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:38:39.959431+0000', 'modified': '2024-09-10T22:38:42.976687+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5304': {'gid': 5304, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.66:6835/3999038135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6833', 'nonce': 3999038135}, {'type': 'v1', 'addr': '172.21.15.66:6835', 'nonce': 3999038135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5304, 'qdb_cluster': [5304]} 2024-09-10T22:38:48.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:38:48.354 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.471+0000 7f222b594640 1 -- 172.21.15.66:0/602027814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22240cf150 msgr2=0x7f22240cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.471+0000 7f222b594640 1 --2- 172.21.15.66:0/602027814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22240cf150 0x7f22240cf590 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2218009900 tx=0x7f221802a580 comp rx=0 tx=0).stop 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.471+0000 7f222b594640 1 -- 172.21.15.66:0/602027814 shutdown_connections 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.471+0000 7f222b594640 1 --2- 172.21.15.66:0/602027814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22240cf150 0x7f22240cf590 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.471+0000 7f222b594640 1 --2- 172.21.15.66:0/602027814 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f222415e310 0x7f22240cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.471+0000 7f222b594640 1 -- 172.21.15.66:0/602027814 >> 172.21.15.66:0/602027814 conn(0x7f22240d4100 msgr2=0x7f22240d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.472+0000 7f222b594640 1 -- 172.21.15.66:0/602027814 shutdown_connections 2024-09-10T22:38:48.473 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.472+0000 7f222b594640 1 -- 172.21.15.66:0/602027814 wait complete. 2024-09-10T22:38:48.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.473+0000 7f222b594640 1 Processor -- start 2024-09-10T22:38:48.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.473+0000 7f222b594640 1 -- start start 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f222b594640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22240cf150 0x7f22240cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f222b594640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22240d0840 0x7f222414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f222b594640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f222415e310 0x7f22241e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f222b594640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f22240b53f0 con 0x7f222415e310 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f222b594640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f22240b5270 con 0x7f22240cf150 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f222b594640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f22240b5570 con 0x7f22240d0840 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f222415e310 0x7f22241e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f222415e310 0x7f22241e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48728/0 (socket says 172.21.15.66:48728) 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 -- 172.21.15.66:0/1839600841 learned_addr learned my addr 172.21.15.66:0/1839600841 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2228b2f640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22240d0840 0x7f222414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229330640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22240cf150 0x7f22240cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 -- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22240d0840 msgr2=0x7f222414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22240d0840 0x7f222414d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 -- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22240cf150 msgr2=0x7f22240cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22240cf150 0x7f22240cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 -- 172.21.15.66:0/1839600841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222414da50 con 0x7f222415e310 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2228b2f640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22240d0840 0x7f222414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229330640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22240cf150 0x7f22240cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2229b31640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f222415e310 0x7f22241e6a20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f221400c770 tx=0x7f221400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2221ffb640 1 -- 172.21.15.66:0/1839600841 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214017070 con 0x7f222415e310 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2221ffb640 1 -- 172.21.15.66:0/1839600841 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2214005070 con 0x7f222415e310 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f222b594640 1 -- 172.21.15.66:0/1839600841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22240cb520 con 0x7f222415e310 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.474+0000 7f2221ffb640 1 -- 172.21.15.66:0/1839600841 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140054c0 con 0x7f222415e310 2024-09-10T22:38:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.475+0000 7f222b594640 1 -- 172.21.15.66:0/1839600841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22240cbae0 con 0x7f222415e310 2024-09-10T22:38:48.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.476+0000 7f2221ffb640 1 -- 172.21.15.66:0/1839600841 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2214005660 con 0x7f222415e310 2024-09-10T22:38:48.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.477+0000 7f2221ffb640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2210077f70 0x7f221007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.477+0000 7f2221ffb640 1 -- 172.21.15.66:0/1839600841 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2214099680 con 0x7f222415e310 2024-09-10T22:38:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.477+0000 7f2229330640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2210077f70 0x7f221007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.477+0000 7f222e43d740 1 Processor -- start 2024-09-10T22:38:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.477+0000 7f222e43d740 1 -- start start 2024-09-10T22:38:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f2229330640 1 --2- 172.21.15.66:0/1839600841 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2210077f70 0x7f221007a430 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f221c005fd0 tx=0x7f221c00f360 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f222e43d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f23ea1b0 0x55d2f23c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f222e43d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2409df0 0x55d2f240a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f222e43d740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23bf5f0 0x55d2f23bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f222e43d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2f2336820 con 0x55d2f2409df0 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f222e43d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d2f2325c50 con 0x55d2f23ea1b0 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f222e43d740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55d2f2329a60 con 0x55d2f23bf5f0 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f2228b2f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2409df0 0x55d2f240a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f2228b2f640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2409df0 0x55d2f240a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48740/0 (socket says 172.21.15.66:48740) 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f2228b2f640 1 -- 172.21.15.66:0/96058792 learned_addr learned my addr 172.21.15.66:0/96058792 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f2229b31640 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23bf5f0 0x55d2f23bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.478+0000 7f2229330640 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f23ea1b0 0x55d2f23c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f2228b2f640 1 -- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23bf5f0 msgr2=0x55d2f23bfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f2228b2f640 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23bf5f0 0x55d2f23bfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f2228b2f640 1 -- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f23ea1b0 msgr2=0x55d2f23c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f2228b2f640 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f23ea1b0 0x55d2f23c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f2228b2f640 1 -- 172.21.15.66:0/96058792 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f2087850 con 0x55d2f2409df0 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f2228b2f640 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2409df0 0x55d2f240a1f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2218009a20 tx=0x7f22180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573991d9ac481716 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f220affd640 1 -- 172.21.15.66:0/96058792 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221803f070 con 0x55d2f2409df0 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f220affd640 1 -- 172.21.15.66:0/96058792 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2218005020 con 0x55d2f2409df0 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f220affd640 1 -- 172.21.15.66:0/96058792 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218035070 con 0x55d2f2409df0 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f222e43d740 1 -- 172.21.15.66:0/96058792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2409df0 msgr2=0x55d2f240a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f222e43d740 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2409df0 0x55d2f240a1f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2218009a20 tx=0x7f22180046c0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f222e43d740 1 -- 172.21.15.66:0/96058792 shutdown_connections 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f222e43d740 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23bf5f0 0x55d2f23bfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f222e43d740 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2409df0 0x55d2f240a1f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f222e43d740 1 --2- 172.21.15.66:0/96058792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f23ea1b0 0x55d2f23c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.479+0000 7f222e43d740 1 -- 172.21.15.66:0/96058792 >> 172.21.15.66:0/96058792 conn(0x55d2f23c06a0 msgr2=0x55d2f23c40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:38:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.480+0000 7f222e43d740 1 -- 172.21.15.66:0/96058792 shutdown_connections 2024-09-10T22:38:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.480+0000 7f222e43d740 1 -- 172.21.15.66:0/96058792 wait complete. 2024-09-10T22:38:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.480+0000 7f222e43d740 1 -- start start 2024-09-10T22:38:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.480+0000 7f222e43d740 1 Processor -- start 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f222e43d740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23ea1b0 0x55d2f23e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f222e43d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f2409df0 0x55d2f24195f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f222e43d740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2419d90 0x55d2f2402490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f222e43d740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x55d2f2096620 con 0x55d2f2419d90 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f222e43d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55d2f22cb5f0 con 0x55d2f2409df0 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f222e43d740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x55d2f23beb30 con 0x55d2f23ea1b0 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f222e43d740 10 client.0 _ll_register_callbacks cb 0x55d2f22cf0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f2229b31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2419d90 0x55d2f2402490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.487 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f2229b31640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2419d90 0x55d2f2402490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48742/0 (socket says 172.21.15.66:48742) 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f2229b31640 1 -- 172.21.15.66:0/3604288841 learned_addr learned my addr 172.21.15.66:0/3604288841 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23ea1b0 0x55d2f23e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.481+0000 7f2228b2f640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f2409df0 0x55d2f24195f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229b31640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23ea1b0 msgr2=0x55d2f23e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229b31640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23ea1b0 0x55d2f23e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229b31640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f2409df0 msgr2=0x55d2f24195f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229b31640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f2409df0 0x55d2f24195f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229b31640 1 -- 172.21.15.66:0/3604288841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f20fe6b0 con 0x55d2f2419d90 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2228b2f640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d2f2409df0 0x55d2f24195f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x55d2f23ea1b0 0x55d2f23e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229b31640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2f2419d90 0x55d2f2402490 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f221405ed80 tx=0x7f2214062130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b3f02e97ed90b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2203fff640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2203fff640 1 -- 172.21.15.66:0/3604288841 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214061b10 con 0x55d2f2419d90 2024-09-10T22:38:48.488 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2203fff640 1 -- 172.21.15.66:0/3604288841 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2214060d70 con 0x55d2f2419d90 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 1 -- 172.21.15.66:0/3604288841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2218005020 con 0x55d2f2419d90 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2203fff640 1 -- 172.21.15.66:0/3604288841 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f22140665f0 con 0x55d2f2419d90 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 10 client.5475 fetch_fsmap learned FSMap version 32 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 1 -- 172.21.15.66:0/3604288841 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55d2f24003b0 con 0x55d2f2419d90 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2203fff640 1 -- 172.21.15.66:0/3604288841 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2218005020 con 0x55d2f2419d90 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2203fff640 1 -- 172.21.15.66:0/3604288841 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) ==== 3502+0+0 (secure 0 0 0) 0x7f221405d460 con 0x55d2f2419d90 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 10 client.5475 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 10 client.5475 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 20 client.5475 populate_metadata read hostname 'smithi066' 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 4 client.5475 mds_command: new command op to 5304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f222e43d740 1 -- 172.21.15.66:0/3604288841 --> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2f2338770 con 0x55d2f241c570 2024-09-10T22:38:48.489 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.482+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.483+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:38:48.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.483+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:38:48.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.483+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:38:48.490 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.483+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T22:38:48.644 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:38:48.643+0000 7fe21bcb1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:38:48.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.684+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:48.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.685+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:38:48.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.685+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:38:48.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.685+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:38:48.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:48.685+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T22:38:49.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.087+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:49.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.087+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:38:49.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.087+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:38:49.088 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.087+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:38:49.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.087+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T22:38:49.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.888+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:49.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.889+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:38:49.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.889+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:38:49.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.889+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:38:49.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:49.889+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T22:38:51.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:51.491+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:51.492 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:51.491+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:38:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:51.491+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:38:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:51.491+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:38:51.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:51.491+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T22:38:53.654 INFO:tasks.ceph.mds.c.smithi066.stderr:2024-09-10T22:38:53.653+0000 7f371001e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:38:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:54.695+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:38:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:54.696+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:38:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:54.696+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:38:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:54.696+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:38:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:38:54.696+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-09-10T22:39:01.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:01.102+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:01.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:01.103+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:39:01.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:01.103+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:39:01.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:01.103+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:39:01.104 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:01.103+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-09-10T22:39:02.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:39:02.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:39:13.917 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:13.916+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:13.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:13.917+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:39:13.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:13.917+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:39:13.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:13.917+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:39:13.918 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:13.917+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:39:28.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:28.931+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:28.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:28.931+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:39:28.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:28.931+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:39:28.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:28.931+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:39:28.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:28.931+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:39:32.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:39:32.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:39:43.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:43.947+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:43.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:43.947+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:39:43.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:43.947+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:39:43.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:43.947+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:39:43.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:43.947+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:39:58.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:58.957+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:39:58.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:58.957+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:39:58.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:58.957+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:39:58.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:58.957+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:39:58.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:39:58.957+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:40:02.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:40:02.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:40:13.968 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:13.966+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:13.966+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:40:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:13.966+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:40:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:13.966+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:40:13.969 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:13.966+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:40:28.983 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:28.981+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:28.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:28.981+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:40:28.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:28.981+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:40:28.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:28.981+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:40:28.984 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:28.981+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:40:32.824 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:40:32.827 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:40:44.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:43.997+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:44.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:43.997+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-10T22:40:44.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:43.997+0000 7f2229330640 1 -- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 msgr2=0x55d2f241c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T22:40:44.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:43.997+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T22:40:44.000 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:43.997+0000 7f2229330640 1 --2- 172.21.15.66:0/3604288841 >> [v2:172.21.15.66:6833/3999038135,v1:172.21.15.66:6835/3999038135] conn(0x55d2f241c570 0x55d2f241c950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T22:40:48.387 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-09-10T22:40:49.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.505+0000 7f98a9ff8640 1 -- 172.21.15.66:0/238745097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 msgr2=0x7f98a40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.505+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/238745097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a40b4610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f989c005fd0 tx=0x7f989c02ab40 comp rx=0 tx=0).stop 2024-09-10T22:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.505+0000 7f98a9ff8640 1 -- 172.21.15.66:0/238745097 shutdown_connections 2024-09-10T22:40:49.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.505+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/238745097 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a40b4610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.505+0000 7f98a9ff8640 1 -- 172.21.15.66:0/238745097 >> 172.21.15.66:0/238745097 conn(0x7f98a40b3af0 msgr2=0x7f98a40b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.505+0000 7f98a9ff8640 1 -- 172.21.15.66:0/238745097 shutdown_connections 2024-09-10T22:40:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.505+0000 7f98a9ff8640 1 -- 172.21.15.66:0/238745097 wait complete. 2024-09-10T22:40:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.506+0000 7f98a9ff8640 1 Processor -- start 2024-09-10T22:40:49.508 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.506+0000 7f98a9ff8640 1 -- start start 2024-09-10T22:40:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a9ff8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98a415be30 0x7f98a4153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a9ff8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98a415c210 0x7f98a40cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a9ff8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a4156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a9ff8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40b51e0 con 0x7f98a4160270 2024-09-10T22:40:49.510 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a9ff8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40b5060 con 0x7f98a415c210 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a9ff8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f98a40b5360 con 0x7f98a415be30 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a4156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a4156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57854/0 (socket says 172.21.15.66:57854) 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a3fff640 1 -- 172.21.15.66:0/875180185 learned_addr learned my addr 172.21.15.66:0/875180185 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a37fe640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98a415be30 0x7f98a4153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.507+0000 7f98a2ffd640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98a415c210 0x7f98a40cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a3fff640 1 -- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98a415be30 msgr2=0x7f98a4153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a3fff640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98a415be30 0x7f98a4153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a3fff640 1 -- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98a415c210 msgr2=0x7f98a40cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a3fff640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98a415c210 0x7f98a40cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a3fff640 1 -- 172.21.15.66:0/875180185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a4154440 con 0x7f98a4160270 2024-09-10T22:40:49.511 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a3fff640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a4156410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f989c006010 tx=0x7f989c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:49.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f988bfff640 1 -- 172.21.15.66:0/875180185 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c044070 con 0x7f98a4160270 2024-09-10T22:40:49.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f988bfff640 1 -- 172.21.15.66:0/875180185 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f989c004250 con 0x7f98a4160270 2024-09-10T22:40:49.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a4154720 con 0x7f98a4160270 2024-09-10T22:40:49.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f988bfff640 1 -- 172.21.15.66:0/875180185 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c0046a0 con 0x7f98a4160270 2024-09-10T22:40:49.512 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.508+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a4154fc0 con 0x7f98a4160270 2024-09-10T22:40:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.509+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a40b5810 con 0x7f98a4160270 2024-09-10T22:40:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.510+0000 7f988bfff640 1 -- 172.21.15.66:0/875180185 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f989c02c9a0 con 0x7f98a4160270 2024-09-10T22:40:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.510+0000 7f988bfff640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9890077f70 0x7f989007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.510+0000 7f988bfff640 1 -- 172.21.15.66:0/875180185 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f989c0ba650 con 0x7f98a4160270 2024-09-10T22:40:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.511+0000 7f98a37fe640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9890077f70 0x7f989007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:49.513 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.511+0000 7f98a37fe640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9890077f70 0x7f989007a430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9894000f60 tx=0x7f9894009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:49.516 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.515+0000 7f988bfff640 1 -- 172.21.15.66:0/875180185 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f989c032030 con 0x7f98a4160270 2024-09-10T22:40:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.736+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98a41548f0 con 0x7f98a4160270 2024-09-10T22:40:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.736+0000 7f988bfff640 1 -- 172.21.15.66:0/875180185 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f989c082960 con 0x7f98a4160270 2024-09-10T22:40:49.741 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:49.741 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9890077f70 msgr2=0x7f989007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9890077f70 0x7f989007a430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9894000f60 tx=0x7f9894009040 comp rx=0 tx=0).stop 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 msgr2=0x7f98a4156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a4156410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f989c006010 tx=0x7f989c02c8b0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 shutdown_connections 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f9890077f70 0x7f989007a430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f98a4160270 0x7f98a4156410 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f98a415c210 0x7f98a40cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.742 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 --2- 172.21.15.66:0/875180185 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f98a415be30 0x7f98a4153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 >> 172.21.15.66:0/875180185 conn(0x7f98a40b3af0 msgr2=0x7f98a40b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 shutdown_connections 2024-09-10T22:40:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:49.739+0000 7f98a9ff8640 1 -- 172.21.15.66:0/875180185 wait complete. 2024-09-10T22:40:50.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:38:39.959431+0000', 'modified': '2024-09-10T22:38:52.670633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 50, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5454': {'gid': 5454, 'name': 'c', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.66:6834/917223506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.66:6832', 'nonce': 917223506}, {'type': 'v1', 'addr': '172.21.15.66:6834', 'nonce': 917223506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5454, 'qdb_cluster': [5454]} 2024-09-10T22:40:50.155 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:40:50.155 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 -- 172.21.15.66:0/1782887280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc15e310 msgr2=0x7efddc0cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 --2- 172.21.15.66:0/1782887280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc15e310 0x7efddc0cec10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009830 tx=0x7efdcc02a480 comp rx=0 tx=0).stop 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 -- 172.21.15.66:0/1782887280 shutdown_connections 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 --2- 172.21.15.66:0/1782887280 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efddc0cf150 0x7efddc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 --2- 172.21.15.66:0/1782887280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc15e310 0x7efddc0cec10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 -- 172.21.15.66:0/1782887280 >> 172.21.15.66:0/1782887280 conn(0x7efddc0d4100 msgr2=0x7efddc0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 -- 172.21.15.66:0/1782887280 shutdown_connections 2024-09-10T22:40:50.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.260+0000 7efde134b640 1 -- 172.21.15.66:0/1782887280 wait complete. 2024-09-10T22:40:50.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.261+0000 7efde134b640 1 Processor -- start 2024-09-10T22:40:50.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.261+0000 7efde134b640 1 -- start start 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efde134b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efddc0cf150 0x7efddc1f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efde134b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc0d0840 0x7efddc1f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efde134b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efddc15e310 0x7efddc1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efde134b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0d4f00 con 0x7efddc0d0840 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efde134b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0b5480 con 0x7efddc0cf150 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efde134b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efddc0d13e0 con 0x7efddc15e310 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc0d0840 0x7efddc1f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc0d0840 0x7efddc1f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57884/0 (socket says 172.21.15.66:57884) 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 -- 172.21.15.66:0/696118205 learned_addr learned my addr 172.21.15.66:0/696118205 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efddaffd640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efddc0cf150 0x7efddc1f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efddb7fe640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efddc15e310 0x7efddc1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 -- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efddc15e310 msgr2=0x7efddc1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efddc15e310 0x7efddc1575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 -- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efddc0cf150 msgr2=0x7efddc1f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efddc0cf150 0x7efddc1f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.262+0000 7efdda7fc640 1 -- 172.21.15.66:0/696118205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc0c8300 con 0x7efddc0d0840 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.263+0000 7efdda7fc640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc0d0840 0x7efddc1f4760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c890 tx=0x7efdd000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.263+0000 7efdc37fe640 1 -- 172.21.15.66:0/696118205 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0004220 con 0x7efddc0d0840 2024-09-10T22:40:50.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.263+0000 7efdc37fe640 1 -- 172.21.15.66:0/696118205 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdd00079b0 con 0x7efddc0d0840 2024-09-10T22:40:50.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.263+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc1f4ee0 con 0x7efddc0d0840 2024-09-10T22:40:50.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.263+0000 7efdc37fe640 1 -- 172.21.15.66:0/696118205 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0007de0 con 0x7efddc0d0840 2024-09-10T22:40:50.267 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.263+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc1f54a0 con 0x7efddc0d0840 2024-09-10T22:40:50.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.265+0000 7efdc37fe640 1 -- 172.21.15.66:0/696118205 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efdd0005020 con 0x7efddc0d0840 2024-09-10T22:40:50.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.265+0000 7efdc37fe640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efdc8077f70 0x7efdc807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.265+0000 7efdc37fe640 1 -- 172.21.15.66:0/696118205 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7efdd009a200 con 0x7efddc0d0840 2024-09-10T22:40:50.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.265+0000 7efddaffd640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efdc8077f70 0x7efdc807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.266+0000 7efde41f2740 1 Processor -- start 2024-09-10T22:40:50.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.266+0000 7efde41f2740 1 -- start start 2024-09-10T22:40:50.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.266+0000 7efddaffd640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efdc8077f70 0x7efdc807a430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efdcc034000 tx=0x7efdcc004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efde41f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 0x5562d45f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45a55f0 0x5562d45a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efde41f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5562d451c820 con 0x5562d45d01b0 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5562d450bc50 con 0x5562d45a55f0 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x5562d450fa60 con 0x5562d45efdf0 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45a7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57900/0 (socket says 172.21.15.66:57900) 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 -- 172.21.15.66:0/805587815 learned_addr learned my addr 172.21.15.66:0/805587815 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddb7fe640 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45a55f0 0x5562d45a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.275 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efdda7fc640 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 0x5562d45f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 -- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 msgr2=0x5562d45f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 0x5562d45f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 -- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45a55f0 msgr2=0x5562d45a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45a55f0 0x5562d45a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 -- 172.21.15.66:0/805587815 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562d426d850 con 0x5562d45d01b0 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddb7fe640 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45a55f0 0x5562d45a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efddaffd640 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45a7b20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009920 tx=0x7efdcc03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34829e8e04e53835 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efdc0ff9640 1 -- 172.21.15.66:0/805587815 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc04d070 con 0x5562d45d01b0 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efdc0ff9640 1 -- 172.21.15.66:0/805587815 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdcc049450 con 0x5562d45d01b0 2024-09-10T22:40:50.276 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.267+0000 7efdc0ff9640 1 -- 172.21.15.66:0/805587815 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc049840 con 0x5562d45d01b0 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 -- 172.21.15.66:0/805587815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 msgr2=0x5562d45a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45a7b20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009920 tx=0x7efdcc03e070 comp rx=0 tx=0).stop 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 -- 172.21.15.66:0/805587815 shutdown_connections 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45a55f0 0x5562d45a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 0x5562d45f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 --2- 172.21.15.66:0/805587815 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45a7b20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 -- 172.21.15.66:0/805587815 >> 172.21.15.66:0/805587815 conn(0x5562d45a66a0 msgr2=0x5562d45aa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 -- 172.21.15.66:0/805587815 shutdown_connections 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.268+0000 7efde41f2740 1 -- 172.21.15.66:0/805587815 wait complete. 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.269+0000 7efde41f2740 1 -- start start 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.269+0000 7efde41f2740 1 Processor -- start 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.269+0000 7efde41f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.269+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45efdf0 0x5562d45e8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.269+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 0x5562d45cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.269+0000 7efde41f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5562d427c620 con 0x5562d45d01b0 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5562d44b15f0 con 0x5562d45efdf0 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x5562d45a4b30 con 0x5562d45cc210 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efde41f2740 10 client.0 _ll_register_callbacks cb 0x5562d44b50a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45ba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57910/0 (socket says 172.21.15.66:57910) 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 -- 172.21.15.66:0/424792194 learned_addr learned my addr 172.21.15.66:0/424792194 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddb7fe640 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 0x5562d45cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efdda7fc640 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45efdf0 0x5562d45e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 -- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 msgr2=0x5562d45cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 0x5562d45cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 -- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45efdf0 msgr2=0x5562d45e8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45efdf0 0x5562d45e8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 -- 172.21.15.66:0/424792194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562d42e46b0 con 0x5562d45d01b0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efddaffd640 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45ba3c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efdcc003ad0 tx=0x7efdcc04abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d374a584506e0f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efdb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efdb9ffb640 1 -- 172.21.15.66:0/424792194 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc04d070 con 0x5562d45d01b0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efdb9ffb640 1 -- 172.21.15.66:0/424792194 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7efdcc052660 con 0x5562d45d01b0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdcc049450 con 0x5562d45d01b0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efdb9ffb640 1 -- 172.21.15.66:0/424792194 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efdcc063050 con 0x5562d45d01b0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efde41f2740 10 client.5499 fetch_fsmap learned FSMap version 39 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.270+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5562d4602350 con 0x5562d45d01b0 2024-09-10T22:40:50.279 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efdb9ffb640 1 -- 172.21.15.66:0/424792194 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdcc049450 con 0x5562d45d01b0 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efdb9ffb640 1 -- 172.21.15.66:0/424792194 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 39) ==== 3502+0+0 (secure 0 0 0) 0x7efdcc066020 con 0x5562d45d01b0 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efde41f2740 10 client.5499 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efde41f2740 10 client.5499 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efde41f2740 20 client.5499 populate_metadata read hostname 'smithi066' 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efde41f2740 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45fea80 0x5562d45fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efde41f2740 4 client.5499 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 --> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562d451e770 con 0x5562d45fea80 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efdda7fc640 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45fea80 0x5562d45fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efdda7fc640 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45fea80 0x5562d45fee60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451cb3c2aa49cb36 server_cookie=95da3eb67f2fe982 in_seq=0 out_seq=0 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.271+0000 7efdb9ffb640 10 client.5499 ms_handle_connect on v2:172.21.15.66:6832/917223506 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efdb9ffb640 1 -- 172.21.15.66:0/424792194 <== mds.0 v2:172.21.15.66:6832/917223506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5562d451e770 con 0x5562d45fea80 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efdb9ffb640 10 client.5499 handle_command_reply: tid=0 2024-09-10T22:40:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 client.5499 shutdown 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 msgr2=0x5562d45ba3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45ba3c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efdcc003ad0 tx=0x7efdcc04abb0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 shutdown_connections 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45fea80 0x5562d45fee60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 0x5562d45cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45efdf0 0x5562d45e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 --2- 172.21.15.66:0/424792194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45ba3c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efdb9ffb640 0 client.5499 ms_handle_reset on v2:172.21.15.66:6832/917223506 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 >> 172.21.15.66:0/424792194 conn(0x5562d45a55f0 msgr2=0x5562d45a9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 shutdown_connections 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 1 -- 172.21.15.66:0/424792194 wait complete. 2024-09-10T22:40:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.272+0000 7efde41f2740 20 client.5499 trim_cache size 0 max 16384 2024-09-10T22:40:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.291+0000 7efde41f2740 1 Processor -- start 2024-09-10T22:40:50.293 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.291+0000 7efde41f2740 1 -- start start 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 0x5562d45e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45d01b0 0x5562d45ca930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efde41f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45efdf0 0x5562d45cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efde41f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efddc200330 con 0x5562d45efdf0 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0cbf70 con 0x5562d45d01b0 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efddc06ee60 con 0x5562d45cc210 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45efdf0 0x5562d45cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45efdf0 0x5562d45cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57918/0 (socket says 172.21.15.66:57918) 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 -- 172.21.15.66:0/2318580399 learned_addr learned my addr 172.21.15.66:0/2318580399 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 -- 172.21.15.66:0/2318580399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 msgr2=0x5562d45e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 --2- 172.21.15.66:0/2318580399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 0x5562d45e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 -- 172.21.15.66:0/2318580399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45d01b0 msgr2=0x5562d45ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 --2- 172.21.15.66:0/2318580399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45d01b0 0x5562d45ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 -- 172.21.15.66:0/2318580399 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562d46027b0 con 0x5562d45efdf0 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efddb7fe640 1 --2- 172.21.15.66:0/2318580399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45efdf0 0x5562d45cae70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efdd4003fa0 tx=0x7efdd40054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16b6ad840bc6f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efdbbfff640 1 -- 172.21.15.66:0/2318580399 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd401a070 con 0x5562d45efdf0 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efdbbfff640 1 -- 172.21.15.66:0/2318580399 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdd4001e80 con 0x5562d45efdf0 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.292+0000 7efdbbfff640 1 -- 172.21.15.66:0/2318580399 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4016420 con 0x5562d45efdf0 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 -- 172.21.15.66:0/2318580399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45efdf0 msgr2=0x5562d45cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 --2- 172.21.15.66:0/2318580399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45efdf0 0x5562d45cae70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efdd4003fa0 tx=0x7efdd40054e0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 -- 172.21.15.66:0/2318580399 shutdown_connections 2024-09-10T22:40:50.297 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 --2- 172.21.15.66:0/2318580399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45efdf0 0x5562d45cae70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 --2- 172.21.15.66:0/2318580399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45d01b0 0x5562d45ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 --2- 172.21.15.66:0/2318580399 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45cc210 0x5562d45e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 -- 172.21.15.66:0/2318580399 >> 172.21.15.66:0/2318580399 conn(0x5562d45a55f0 msgr2=0x5562d459fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 -- 172.21.15.66:0/2318580399 shutdown_connections 2024-09-10T22:40:50.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 -- 172.21.15.66:0/2318580399 wait complete. 2024-09-10T22:40:50.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 -- start start 2024-09-10T22:40:50.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.293+0000 7efde41f2740 1 Processor -- start 2024-09-10T22:40:50.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45cc210 0x5562d466f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efde41f2740 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.301 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efde41f2740 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 0x5562d4611430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efde41f2740 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x5562d45ca550 con 0x5562d45d01b0 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5562d4602630 con 0x5562d45cc210 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efde41f2740 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x5562d45c9990 con 0x5562d45efdf0 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efde41f2740 10 client.0 _ll_register_callbacks cb 0x7efdcc004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efdda7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efdda7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57932/0 (socket says 172.21.15.66:57932) 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.294+0000 7efdda7fc640 1 -- 172.21.15.66:0/1810348450 learned_addr learned my addr 172.21.15.66:0/1810348450 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdda7fc640 1 -- 172.21.15.66:0/1810348450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 msgr2=0x5562d4611430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdda7fc640 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 0x5562d4611430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdda7fc640 1 -- 172.21.15.66:0/1810348450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45cc210 msgr2=0x5562d466f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdda7fc640 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45cc210 0x5562d466f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdda7fc640 1 -- 172.21.15.66:0/1810348450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562d4613190 con 0x5562d45d01b0 2024-09-10T22:40:50.302 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdda7fc640 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45d0ae0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efdd005f890 tx=0x7efdd005ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33ec14aed1005bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdb9ffb640 1 -- 172.21.15.66:0/1810348450 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd004d070 con 0x5562d45d01b0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdb9ffb640 1 -- 172.21.15.66:0/1810348450 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7efdd0088120 con 0x5562d45d01b0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd4001e80 con 0x5562d45d01b0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdb9ffb640 1 -- 172.21.15.66:0/1810348450 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efdd001d030 con 0x5562d45d01b0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 10 client.5505 fetch_fsmap learned FSMap version 39 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x5562d45cb210 con 0x5562d45d01b0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdb9ffb640 1 -- 172.21.15.66:0/1810348450 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdd4001e80 con 0x5562d45d01b0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efdb9ffb640 1 -- 172.21.15.66:0/1810348450 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 39) ==== 3502+0+0 (secure 0 0 0) 0x7efdd00840c0 con 0x5562d45d01b0 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 10 client.5505 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 10 client.5505 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 20 client.5505 populate_metadata read hostname 'smithi066' 2024-09-10T22:40:50.303 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45c85a0 0x5562d45ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 4 client.5505 mds_command: new command op to 5454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.295+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 --> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] -- command(tid 0: {"prefix": "status"}) -- 0x5562d45e9e80 con 0x5562d45c85a0 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efddaffd640 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45c85a0 0x5562d45ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efddaffd640 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45c85a0 0x5562d45ee330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622a1e02ed89a128 server_cookie=f054db0fe4a44b48 in_seq=0 out_seq=0 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efdb9ffb640 10 client.5505 ms_handle_connect on v2:172.21.15.66:6832/917223506 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efdb9ffb640 1 -- 172.21.15.66:0/1810348450 <== mds.0 v2:172.21.15.66:6832/917223506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5562d45e9e80 con 0x5562d45c85a0 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efdb9ffb640 10 client.5505 handle_command_reply: tid=0 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 client.5505 shutdown 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 msgr2=0x5562d45d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45d0ae0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efdd005f890 tx=0x7efdd005ecf0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 shutdown_connections 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:6832/917223506,v1:172.21.15.66:6834/917223506] conn(0x5562d45c85a0 0x5562d45ee330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x5562d45efdf0 0x5562d4611430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5562d45d01b0 0x5562d45d0ae0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.304 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 --2- 172.21.15.66:0/1810348450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5562d45cc210 0x5562d466f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efdb9ffb640 0 client.5505 ms_handle_reset on v2:172.21.15.66:6832/917223506 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 >> 172.21.15.66:0/1810348450 conn(0x5562d45a55f0 msgr2=0x5562d45feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 shutdown_connections 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 1 -- 172.21.15.66:0/1810348450 wait complete. 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.296+0000 7efde41f2740 20 client.5505 trim_cache size 0 max 16384 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "7ef137cd-a1ce-4819-b27d-b71268612f85", 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 5454, 2024-09-10T22:40:50.305 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 121.64465500999999, 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 39, 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 53, 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 51, 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 122.622456224 2024-09-10T22:40:50.306 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efdc8077f70 msgr2=0x7efdc807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efdc8077f70 0x7efdc807a430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efdcc034000 tx=0x7efdcc004730 comp rx=0 tx=0).stop 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc0d0840 msgr2=0x7efddc1f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc0d0840 0x7efddc1f4760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c890 tx=0x7efdd000cd50 comp rx=0 tx=0).stop 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 shutdown_connections 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efdc8077f70 0x7efdc807a430 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efddc15e310 0x7efddc1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efddc0d0840 0x7efddc1f4760 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.307 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 --2- 172.21.15.66:0/696118205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efddc0cf150 0x7efddc1f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 >> 172.21.15.66:0/696118205 conn(0x7efddc0d4100 msgr2=0x7efddc0b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 shutdown_connections 2024-09-10T22:40:50.308 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.297+0000 7efde134b640 1 -- 172.21.15.66:0/696118205 wait complete. 2024-09-10T22:40:50.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.412+0000 7f64ae964640 1 -- 172.21.15.66:0/2669936489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a815e2b0 msgr2=0x7f64a80d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.412+0000 7f64ae964640 1 --2- 172.21.15.66:0/2669936489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a815e2b0 0x7f64a80d27a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f649c005fd0 tx=0x7f649c02a5d0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.412+0000 7f64ae964640 1 -- 172.21.15.66:0/2669936489 shutdown_connections 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.413+0000 7f64ae964640 1 --2- 172.21.15.66:0/2669936489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a815e2b0 0x7f64a80d27a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.413+0000 7f64ae964640 1 --2- 172.21.15.66:0/2669936489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64a80d4000 0x7f64a80d43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.413+0000 7f64ae964640 1 -- 172.21.15.66:0/2669936489 >> 172.21.15.66:0/2669936489 conn(0x7f64a8074c20 msgr2=0x7f64a809d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.413+0000 7f64ae964640 1 -- 172.21.15.66:0/2669936489 shutdown_connections 2024-09-10T22:40:50.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.413+0000 7f64ae964640 1 -- 172.21.15.66:0/2669936489 wait complete. 2024-09-10T22:40:50.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.414+0000 7f64ae964640 1 Processor -- start 2024-09-10T22:40:50.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.414+0000 7f64ae964640 1 -- start start 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64ae964640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a80d2ce0 0x7f64a81f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64ae964640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80d4000 0x7f64a80d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64ae964640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64a815e2b0 0x7f64a80d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64ae964640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f64a80b4a80 con 0x7f64a80d2ce0 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64ae964640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f64a80b4900 con 0x7f64a80d4000 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64ae964640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f64a80b4c00 con 0x7f64a815e2b0 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a80d2ce0 0x7f64a81f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a80d2ce0 0x7f64a81f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57960/0 (socket says 172.21.15.66:57960) 2024-09-10T22:40:50.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 -- 172.21.15.66:0/1550233091 learned_addr learned my addr 172.21.15.66:0/1550233091 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64acf01640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64a815e2b0 0x7f64a80d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a77fe640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80d4000 0x7f64a80d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 -- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64a815e2b0 msgr2=0x7f64a80d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64a815e2b0 0x7f64a80d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 -- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80d4000 msgr2=0x7f64a80d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80d4000 0x7f64a80d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 -- 172.21.15.66:0/1550233091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a8159280 con 0x7f64a80d2ce0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a7fff640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a80d2ce0 0x7f64a81f4760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f649800d980 tx=0x7f649800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a4ff9640 1 -- 172.21.15.66:0/1550233091 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498017070 con 0x7f64a80d2ce0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a4ff9640 1 -- 172.21.15.66:0/1550233091 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6498009d20 con 0x7f64a80d2ce0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a814b020 con 0x7f64a80d2ce0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.415+0000 7f64a4ff9640 1 -- 172.21.15.66:0/1550233091 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64980106c0 con 0x7f64a80d2ce0 2024-09-10T22:40:50.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.416+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a814b5e0 con 0x7f64a80d2ce0 2024-09-10T22:40:50.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.416+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a80d43e0 con 0x7f64a80d2ce0 2024-09-10T22:40:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.417+0000 7f64a4ff9640 1 -- 172.21.15.66:0/1550233091 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6498010860 con 0x7f64a80d2ce0 2024-09-10T22:40:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.418+0000 7f64a4ff9640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6494077f70 0x7f649407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.418+0000 7f64a4ff9640 1 -- 172.21.15.66:0/1550233091 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f649809a0a0 con 0x7f64a80d2ce0 2024-09-10T22:40:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.418+0000 7f64a77fe640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6494077f70 0x7f649407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:50.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.418+0000 7f64a77fe640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6494077f70 0x7f649407a430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f649c033000 tx=0x7f649c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:50.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.422+0000 7f64a4ff9640 1 -- 172.21.15.66:0/1550233091 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6498062550 con 0x7f64a80d2ce0 2024-09-10T22:40:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.642+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64a806eb20 con 0x7f64a80d2ce0 2024-09-10T22:40:50.644 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.642+0000 7f64a4ff9640 1 -- 172.21.15.66:0/1550233091 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f6498066f80 con 0x7f64a80d2ce0 2024-09-10T22:40:50.645 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:50.645 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:50.645 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6494077f70 msgr2=0x7f649407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6494077f70 0x7f649407a430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f649c033000 tx=0x7f649c004730 comp rx=0 tx=0).stop 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a80d2ce0 msgr2=0x7f64a81f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a80d2ce0 0x7f64a81f4760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f649800d980 tx=0x7f649800de40 comp rx=0 tx=0).stop 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 shutdown_connections 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6494077f70 0x7f649407a430 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f64a815e2b0 0x7f64a80d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a80d4000 0x7f64a80d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.647 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 --2- 172.21.15.66:0/1550233091 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a80d2ce0 0x7f64a81f4760 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 >> 172.21.15.66:0/1550233091 conn(0x7f64a8074c20 msgr2=0x7f64a806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 shutdown_connections 2024-09-10T22:40:50.648 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:50.644+0000 7f64ae964640 1 -- 172.21.15.66:0/1550233091 wait complete. 2024-09-10T22:40:50.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-10T22:40:50.776 INFO:teuthology.orchestra.run.smithi066.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":187,"msgr_send_messages":62,"msgr_recv_bytes":182000,"msgr_send_bytes":27665,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.033443830,"msgr_running_send_time":0.011644682,"msgr_running_recv_time":0.014346510,"msgr_running_fast_dispatch_time":0.007443970,"msgr_send_messages_queue_lat":{"avgcount":62,"sum":0.008755116,"avgtime":0.000141211},"msgr_handle_ack_lat":{"avgcount":139,"sum":0.000026414,"avgtime":0.000000190},"msgr_recv_encrypted_bytes":129408,"msgr_send_encrypted_bytes":24864},"AsyncMessenger::Worker-1":{"msgr_recv_messages":23,"msgr_send_messages":50,"msgr_recv_bytes":12576,"msgr_send_bytes":145432,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006199574,"msgr_running_send_time":0.002559919,"msgr_running_recv_time":0.002700449,"msgr_running_fast_dispatch_time":0.000496452,"msgr_send_messages_queue_lat":{"avgcount":49,"sum":0.003748652,"avgtime":0.000076503},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":136176},"AsyncMessenger::Worker-2":{"msgr_recv_messages":24,"msgr_send_messages":26,"msgr_recv_bytes":15107,"msgr_send_bytes":33016,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.004269265,"msgr_running_send_time":0.000821598,"msgr_running_recv_time":0.002647701,"msgr_running_fast_dispatch_time":0.000511810,"msgr_send_messages_queue_lat":{"avgcount":24,"sum":0.007010967,"avgtime":0.000292123},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000279,"avgtime":0.000000139},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":49,"sum":0.003216997,"avgtime":0.000065653}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000108880,"avgtime":0.000027220}},"mds":{"request":2,"reply":0,"reply_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":7,"inodes_bottom":3,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":1,"traverse_hit":1,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":1,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":0},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":2,"evex":0,"evtrm":0,"ev":5,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4196431,"rdpos":4195244,"jlat":{"avgcount":2,"sum":0.004020523,"avgtime":0.002010261},"replayed":3},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":1,"cap-":0,"rss":48020,"heap":280825856},"mds_server":{"dispatch_client_request":1,"dispatch_server_request":0,"handle_client_request":2,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":0,"average_load":0,"avg_session_uptime":767,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":228980,"buffer_anon_items":64,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5736,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52800,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":47,"op_send_bytes":1559,"op_resend":0,"op_reply":47,"op_latency":{"avgcount":47,"sum":0.052999598,"avgtime":0.001127651},"op_inflight":0,"oplen_avg":{"avgcount":47,"sum":94},"op":47,"op_r":36,"op_w":11,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":2,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":53,"map_full":0,"map_inc":2,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":3,"omap_rd":48,"omap_del":0},"oft":{"omap_total_objs":0,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":230,"get_sum":184519,"get_or_fail_fail":0,"get_or_fail_success":230,"take":0,"take_sum":0,"put":230,"put_sum":184519,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":2491,"put":47,"put_sum":2491,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":47,"put":47,"put_sum":47,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55e8d05a6d20":{"val":0,"max":3758096384,"get_started":0,"get":2,"get_sum":931,"get_or_fail_fail":0,"get_or_fail_success":2,"take":0,"take_sum":0,"put":2,"put_sum":931,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-10T22:40:50.778 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 139, "avgtime": 1.9e-07, "sum": 2.6414e-05 }, "msgr_recv_bytes": 182000, "msgr_recv_encrypted_bytes": 129408, "msgr_recv_messages": 187, "msgr_running_fast_dispatch_time": 0.00744397, "msgr_running_recv_time": 0.01434651, "msgr_running_send_time": 0.011644682, "msgr_running_total_time": 0.03344383, "msgr_send_bytes": 27665, "msgr_send_encrypted_bytes": 24864, "msgr_send_messages": 62, "msgr_send_messages_queue_lat": { "avgcount": 62, "avgtime": 0.000141211, "sum": 0.008755116 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 12576, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.000496452, "msgr_running_recv_time": 0.002700449, "msgr_running_send_time": 0.002559919, "msgr_running_total_time": 0.006199574, "msgr_send_bytes": 145432, "msgr_send_encrypted_bytes": 136176, "msgr_send_messages": 50, "msgr_send_messages_queue_lat": { "avgcount": 49, "avgtime": 7.6503e-05, "sum": 0.003748652 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.39e-07, "sum": 2.79e-07 }, "msgr_recv_bytes": 15107, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 24, "msgr_running_fast_dispatch_time": 0.00051181, "msgr_running_recv_time": 0.002647701, "msgr_running_send_time": 0.000821598, "msgr_running_total_time": 0.004269265, "msgr_send_bytes": 33016, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 26, "msgr_send_messages_queue_lat": { "avgcount": 24, "avgtime": 0.000292123, "sum": 0.007010967 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 49, "avgtime": 6.5653e-05, "sum": 0.003216997 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.722e-05, "sum": 0.00010888 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 3, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 7, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 0, "q": 0, "reply": 0, "reply_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "request": 2, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 1, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 1, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 5, "evadd": 2, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 2, "avgtime": 0.002010261, "sum": 0.004020523 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4196431 }, "mds_mem": { "cap": 1, "cap+": 1, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 280825856, "ino": 13, "ino+": 13, "ino-": 0, "rss": 48020 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 1, "dispatch_server_request": 0, "handle_client_request": 2, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 0, "avg_session_uptime": 767, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 0 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 228980, "buffer_anon_items": 64, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52800, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5736, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 53, "map_full": 0, "map_inc": 2, "omap_del": 0, "omap_rd": 48, "omap_wr": 3, "op": 47, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 47, "avgtime": 0.001127651, "sum": 0.052999598 }, "op_pg": 0, "op_r": 36, "op_reply": 47, "op_resend": 0, "op_rmw": 0, "op_send": 47, "op_send_bytes": 1559, "op_w": 11, "oplen_avg": { "avgcount": 47, "sum": 94 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 0, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 230, "get_or_fail_fail": 0, "get_or_fail_success": 230, "get_started": 0, "get_sum": 184519, "max": 104857600, "put": 230, "put_sum": 184519, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 47, "put_sum": 2491, "take": 47, "take_sum": 2491, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 47, "put_sum": 47, "take": 47, "take_sum": 47, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55e8d05a6d20": { "get": 2, "get_or_fail_fail": 0, "get_or_fail_success": 2, "get_started": 0, "get_sum": 931, "max": 3758096384, "put": 2, "put_sum": 931, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-10T22:40:50.778 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_0:0 2024-09-10T22:40:50.778 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:40:50.778 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:40:50.795 DEBUG:teuthology.orchestra.run.smithi066:> stdin-killer -- python3 -c ' 2024-09-10T22:40:50.795 DEBUG:teuthology.orchestra.run.smithi066:> import fcntl 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> import os 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> import sys 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> import time 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> with open("/home/ubuntu/cephtest/mnt.0/omap_counter_test_file", '"'"'w'"'"') as f: 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> f.write("content") 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> f.flush() 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> while True: 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> print("open_background: keeping file open", file=sys.stderr) 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> try: 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> if os.read(0, 4096) == b"": 2024-09-10T22:40:50.796 DEBUG:teuthology.orchestra.run.smithi066:> break 2024-09-10T22:40:50.797 DEBUG:teuthology.orchestra.run.smithi066:> except BlockingIOError: 2024-09-10T22:40:50.797 DEBUG:teuthology.orchestra.run.smithi066:> pass 2024-09-10T22:40:50.797 DEBUG:teuthology.orchestra.run.smithi066:> time.sleep(2) 2024-09-10T22:40:50.797 DEBUG:teuthology.orchestra.run.smithi066:> ' 2024-09-10T22:40:50.838 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-09-10T22:40:50.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:40:50.854 INFO:teuthology.orchestra.run.smithi066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0/omap_counter_test_file': No such file or directory 2024-09-10T22:40:50.909 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:40:51.854 DEBUG:teuthology.orchestra.run.smithi066:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-09-10T22:40:51.874 INFO:teuthology.orchestra.run.smithi066.stdout:7 2024-09-10T22:40:51.874 INFO:tasks.cephfs.mount:File omap_counter_test_file became visible with size 7 from 0 after 1s 2024-09-10T22:40:51.875 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:52.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3100885956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0c56e0 msgr2=0x7f8aac073110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3100885956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0c56e0 0x7f8aac073110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009860 tx=0x7f8a9802a540 comp rx=0 tx=0).stop 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3100885956 shutdown_connections 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3100885956 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac0c6160 0x7f8aac0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3100885956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0c56e0 0x7f8aac073110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3100885956 >> 172.21.15.66:0/3100885956 conn(0x7f8aac071380 msgr2=0x7f8aac06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3100885956 shutdown_connections 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.027+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3100885956 wait complete. 2024-09-10T22:40:52.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.028+0000 7f8ab2fd3640 1 Processor -- start 2024-09-10T22:40:52.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.028+0000 7f8ab2fd3640 1 -- start start 2024-09-10T22:40:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.029+0000 7f8ab2fd3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0b2de0 0x7f8aac0cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.029+0000 7f8ab2fd3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0c56e0 0x7f8aac0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.029+0000 7f8ab2fd3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac0c6160 0x7f8aac13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.032 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.029+0000 7f8ab2fd3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac15c890 con 0x7f8aac0c56e0 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.029+0000 7f8ab2fd3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac15c710 con 0x7f8aac0b2de0 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.029+0000 7f8ab2fd3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f8aac15ca10 con 0x7f8aac0c6160 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0c56e0 0x7f8aac0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0c56e0 0x7f8aac0cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57972/0 (socket says 172.21.15.66:57972) 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 -- 172.21.15.66:0/3878720036 learned_addr learned my addr 172.21.15.66:0/3878720036 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8ab1570640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac0c6160 0x7f8aac13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8ab0d6f640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0b2de0 0x7f8aac0cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 -- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac0c6160 msgr2=0x7f8aac13fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac0c6160 0x7f8aac13fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 -- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0b2de0 msgr2=0x7f8aac0cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0b2de0 0x7f8aac0cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 -- 172.21.15.66:0/3878720036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac1f39b0 con 0x7f8aac0c56e0 2024-09-10T22:40:52.033 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aabfff640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0c56e0 0x7f8aac0cedb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009980 tx=0x7f8a98032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aa97fa640 1 -- 172.21.15.66:0/3878720036 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98040070 con 0x7f8aac0c56e0 2024-09-10T22:40:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aa97fa640 1 -- 172.21.15.66:0/3878720036 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a980038e0 con 0x7f8aac0c56e0 2024-09-10T22:40:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac1f3c80 con 0x7f8aac0c56e0 2024-09-10T22:40:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8aa97fa640 1 -- 172.21.15.66:0/3878720036 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98003d30 con 0x7f8aac0c56e0 2024-09-10T22:40:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.030+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac0cf830 con 0x7f8aac0c56e0 2024-09-10T22:40:52.034 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.032+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac15cec0 con 0x7f8aac0c56e0 2024-09-10T22:40:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.032+0000 7f8aa97fa640 1 -- 172.21.15.66:0/3878720036 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8a98045460 con 0x7f8aac0c56e0 2024-09-10T22:40:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.033+0000 7f8aa97fa640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a90077f70 0x7f8a9007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.033+0000 7f8aa97fa640 1 -- 172.21.15.66:0/3878720036 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f8a980bb650 con 0x7f8aac0c56e0 2024-09-10T22:40:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.033+0000 7f8ab0d6f640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a90077f70 0x7f8a9007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.035 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.033+0000 7f8ab0d6f640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a90077f70 0x7f8a9007a430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0c5da0 tx=0x7f8aa0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:52.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.037+0000 7f8aa97fa640 1 -- 172.21.15.66:0/3878720036 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8aac1f3c80 con 0x7f8aac0c56e0 2024-09-10T22:40:52.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.257+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8aac156250 con 0x7f8aac0c56e0 2024-09-10T22:40:52.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.257+0000 7f8aa97fa640 1 -- 172.21.15.66:0/3878720036 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f8a980840d0 con 0x7f8aac0c56e0 2024-09-10T22:40:52.260 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:52.260 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:52.260 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a90077f70 msgr2=0x7f8a9007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a90077f70 0x7f8a9007a430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0c5da0 tx=0x7f8aa0009040 comp rx=0 tx=0).stop 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0c56e0 msgr2=0x7f8aac0cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0c56e0 0x7f8aac0cedb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009980 tx=0x7f8a98032c10 comp rx=0 tx=0).stop 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 shutdown_connections 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8a90077f70 0x7f8a9007a430 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8aac0c6160 0x7f8aac13fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8aac0c56e0 0x7f8aac0cedb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 --2- 172.21.15.66:0/3878720036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac0b2de0 0x7f8aac0cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 >> 172.21.15.66:0/3878720036 conn(0x7f8aac071380 msgr2=0x7f8aac06f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:52.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 shutdown_connections 2024-09-10T22:40:52.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.260+0000 7f8ab2fd3640 1 -- 172.21.15.66:0/3878720036 wait complete. 2024-09-10T22:40:52.273 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-10T22:40:52.392 INFO:teuthology.orchestra.run.smithi066.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":190,"msgr_send_messages":64,"msgr_recv_bytes":183091,"msgr_send_bytes":29277,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.033740035,"msgr_running_send_time":0.011802606,"msgr_running_recv_time":0.014469765,"msgr_running_fast_dispatch_time":0.007482578,"msgr_send_messages_queue_lat":{"avgcount":64,"sum":0.008813789,"avgtime":0.000137715},"msgr_handle_ack_lat":{"avgcount":142,"sum":0.000026949,"avgtime":0.000000189},"msgr_recv_encrypted_bytes":129680,"msgr_send_encrypted_bytes":25504},"AsyncMessenger::Worker-1":{"msgr_recv_messages":25,"msgr_send_messages":52,"msgr_recv_bytes":13068,"msgr_send_bytes":147947,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006321962,"msgr_running_send_time":0.002612589,"msgr_running_recv_time":0.002727653,"msgr_running_fast_dispatch_time":0.000532521,"msgr_send_messages_queue_lat":{"avgcount":51,"sum":0.003826316,"avgtime":0.000075025},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":136176},"AsyncMessenger::Worker-2":{"msgr_recv_messages":28,"msgr_send_messages":30,"msgr_recv_bytes":16401,"msgr_send_bytes":35106,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.004646581,"msgr_running_send_time":0.001036415,"msgr_running_recv_time":0.002714958,"msgr_running_fast_dispatch_time":0.000586668,"msgr_send_messages_queue_lat":{"avgcount":28,"sum":0.007213381,"avgtime":0.000257620},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000279,"avgtime":0.000000139},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.003600788,"avgtime":0.000066681}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000108880,"avgtime":0.000027220}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":2,"traverse_hit":2,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":1,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":3,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198220,"rdpos":4195244,"jlat":{"avgcount":3,"sum":0.006218651,"avgtime":0.002072883},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":48532,"heap":280825856},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":767,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":228980,"buffer_anon_items":64,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5736,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52800,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":53,"op_send_bytes":3974,"op_resend":0,"op_reply":53,"op_latency":{"avgcount":53,"sum":0.060999538,"avgtime":0.001150934},"op_inflight":0,"oplen_avg":{"avgcount":53,"sum":107},"op":53,"op_r":36,"op_w":17,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":53,"map_full":0,"map_inc":2,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":11,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":239,"get_sum":186534,"get_or_fail_fail":0,"get_or_fail_success":239,"take":0,"take_sum":0,"put":239,"put_sum":186534,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":53,"take_sum":4906,"put":53,"put_sum":4906,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":53,"take_sum":53,"put":53,"put_sum":53,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55e8d05a6d20":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2720,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2720,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-10T22:40:52.394 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 142, "avgtime": 1.89e-07, "sum": 2.6949e-05 }, "msgr_recv_bytes": 183091, "msgr_recv_encrypted_bytes": 129680, "msgr_recv_messages": 190, "msgr_running_fast_dispatch_time": 0.007482578, "msgr_running_recv_time": 0.014469765, "msgr_running_send_time": 0.011802606, "msgr_running_total_time": 0.033740035, "msgr_send_bytes": 29277, "msgr_send_encrypted_bytes": 25504, "msgr_send_messages": 64, "msgr_send_messages_queue_lat": { "avgcount": 64, "avgtime": 0.000137715, "sum": 0.008813789 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 13068, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 25, "msgr_running_fast_dispatch_time": 0.000532521, "msgr_running_recv_time": 0.002727653, "msgr_running_send_time": 0.002612589, "msgr_running_total_time": 0.006321962, "msgr_send_bytes": 147947, "msgr_send_encrypted_bytes": 136176, "msgr_send_messages": 52, "msgr_send_messages_queue_lat": { "avgcount": 51, "avgtime": 7.5025e-05, "sum": 0.003826316 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.39e-07, "sum": 2.79e-07 }, "msgr_recv_bytes": 16401, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 28, "msgr_running_fast_dispatch_time": 0.000586668, "msgr_running_recv_time": 0.002714958, "msgr_running_send_time": 0.001036415, "msgr_running_total_time": 0.004646581, "msgr_send_bytes": 35106, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 28, "avgtime": 0.00025762, "sum": 0.007213381 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 6.6681e-05, "sum": 0.003600788 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.722e-05, "sum": 0.00010888 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "request": 3, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 2, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 2, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 6, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.002072883, "sum": 0.006218651 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4198220 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 280825856, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48532 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 767, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 228980, "buffer_anon_items": 64, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52800, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5736, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 53, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 11, "op": 53, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 53, "avgtime": 0.001150934, "sum": 0.060999538 }, "op_pg": 0, "op_r": 36, "op_reply": 53, "op_resend": 0, "op_rmw": 0, "op_send": 53, "op_send_bytes": 3974, "op_w": 17, "oplen_avg": { "avgcount": 53, "sum": 107 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 239, "get_or_fail_fail": 0, "get_or_fail_success": 239, "get_started": 0, "get_sum": 186534, "max": 104857600, "put": 239, "put_sum": 186534, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 53, "put_sum": 4906, "take": 53, "take_sum": 4906, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 53, "put_sum": 53, "take": 53, "take_sum": 53, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55e8d05a6d20": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2720, "max": 3758096384, "put": 3, "put_sum": 2720, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-10T22:40:52.394 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T22:40:52.394 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.497+0000 7f6144752640 1 -- 172.21.15.66:0/997599666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 msgr2=0x7f6124003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.498+0000 7f6144752640 1 --2- 172.21.15.66:0/997599666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 0x7f6124003dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6134009b30 tx=0x7f6124005830 comp rx=0 tx=0).stop 2024-09-10T22:40:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.498+0000 7f6142cef640 1 -- 172.21.15.66:0/997599666 reap_dead start 2024-09-10T22:40:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.498+0000 7f6144752640 1 -- 172.21.15.66:0/997599666 shutdown_connections 2024-09-10T22:40:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.498+0000 7f6144752640 1 -- 172.21.15.66:0/997599666 >> 172.21.15.66:0/997599666 conn(0x7f613c0b1950 msgr2=0x7f613c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.498+0000 7f6144752640 1 -- 172.21.15.66:0/997599666 shutdown_connections 2024-09-10T22:40:52.500 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.498+0000 7f6144752640 1 -- 172.21.15.66:0/997599666 wait complete. 2024-09-10T22:40:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.499+0000 7f6144752640 1 Processor -- start 2024-09-10T22:40:52.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.499+0000 7f6144752640 1 -- start start 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6144752640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 0x7f613c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6144752640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613c0c7830 0x7f613c154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6144752640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f613c0c7c80 0x7f613c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6144752640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f613c0cc6e0 con 0x7f61240039e0 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6144752640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f613c160780 con 0x7f613c0c7830 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6144752640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f613c0cc860 con 0x7f613c0c7c80 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 0x7f613c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 0x7f613c0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57994/0 (socket says 172.21.15.66:57994) 2024-09-10T22:40:52.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 -- 172.21.15.66:0/3303639224 learned_addr learned my addr 172.21.15.66:0/3303639224 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6141ced640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613c0c7830 0x7f613c154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6142cef640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f613c0c7c80 0x7f613c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 -- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f613c0c7c80 msgr2=0x7f613c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f613c0c7c80 0x7f613c0c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 -- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613c0c7830 msgr2=0x7f613c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613c0c7830 0x7f613c154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 -- 172.21.15.66:0/3303639224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c1f3670 con 0x7f61240039e0 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f61424ee640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 0x7f613c0c9cb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6134009b30 tx=0x7f613402b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:52.504 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f613affd640 1 -- 172.21.15.66:0/3303639224 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134041070 con 0x7f61240039e0 2024-09-10T22:40:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f613affd640 1 -- 172.21.15.66:0/3303639224 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61340047a0 con 0x7f61240039e0 2024-09-10T22:40:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c1f3c60 con 0x7f61240039e0 2024-09-10T22:40:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.500+0000 7f613affd640 1 -- 172.21.15.66:0/3303639224 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134004bf0 con 0x7f61240039e0 2024-09-10T22:40:52.505 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.501+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c154e70 con 0x7f61240039e0 2024-09-10T22:40:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.501+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613c0ccdc0 con 0x7f61240039e0 2024-09-10T22:40:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.502+0000 7f613affd640 1 -- 172.21.15.66:0/3303639224 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6134033440 con 0x7f61240039e0 2024-09-10T22:40:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.503+0000 7f613affd640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6128077fa0 0x7f612807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.503+0000 7f613affd640 1 -- 172.21.15.66:0/3303639224 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f61340bb140 con 0x7f61240039e0 2024-09-10T22:40:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.503+0000 7f6141ced640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6128077fa0 0x7f612807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.503+0000 7f6141ced640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6128077fa0 0x7f612807a460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f613c0b1720 tx=0x7f6130006c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:52.509 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.508+0000 7f613affd640 1 -- 172.21.15.66:0/3303639224 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f61340051f0 con 0x7f61240039e0 2024-09-10T22:40:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.727+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f613c06fcc0 con 0x7f61240039e0 2024-09-10T22:40:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.727+0000 7f613affd640 1 -- 172.21.15.66:0/3303639224 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f6134083bb0 con 0x7f61240039e0 2024-09-10T22:40:52.729 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:52.729 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6128077fa0 msgr2=0x7f612807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6128077fa0 0x7f612807a460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f613c0b1720 tx=0x7f6130006c00 comp rx=0 tx=0).stop 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 msgr2=0x7f613c0c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 0x7f613c0c9cb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6134009b30 tx=0x7f613402b9e0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 shutdown_connections 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6128077fa0 0x7f612807a460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f613c0c7c80 0x7f613c0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613c0c7830 0x7f613c154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 --2- 172.21.15.66:0/3303639224 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61240039e0 0x7f613c0c9cb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.729+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 >> 172.21.15.66:0/3303639224 conn(0x7f613c0b1950 msgr2=0x7f613c0ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.730+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 shutdown_connections 2024-09-10T22:40:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52.730+0000 7f6144752640 1 -- 172.21.15.66:0/3303639224 wait complete. 2024-09-10T22:40:52.742 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-10T22:40:52.861 INFO:teuthology.orchestra.run.smithi066.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":191,"msgr_send_messages":64,"msgr_recv_bytes":183509,"msgr_send_bytes":29277,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.033932400,"msgr_running_send_time":0.011879793,"msgr_running_recv_time":0.014528693,"msgr_running_fast_dispatch_time":0.007532490,"msgr_send_messages_queue_lat":{"avgcount":64,"sum":0.008813789,"avgtime":0.000137715},"msgr_handle_ack_lat":{"avgcount":143,"sum":0.000027074,"avgtime":0.000000189},"msgr_recv_encrypted_bytes":129680,"msgr_send_encrypted_bytes":25504},"AsyncMessenger::Worker-1":{"msgr_recv_messages":25,"msgr_send_messages":52,"msgr_recv_bytes":13068,"msgr_send_bytes":147947,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006321962,"msgr_running_send_time":0.002612589,"msgr_running_recv_time":0.002727653,"msgr_running_fast_dispatch_time":0.000532521,"msgr_send_messages_queue_lat":{"avgcount":51,"sum":0.003826316,"avgtime":0.000075025},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":136176},"AsyncMessenger::Worker-2":{"msgr_recv_messages":28,"msgr_send_messages":30,"msgr_recv_bytes":16401,"msgr_send_bytes":35106,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.004646581,"msgr_running_send_time":0.001036415,"msgr_running_recv_time":0.002714958,"msgr_running_fast_dispatch_time":0.000586668,"msgr_send_messages_queue_lat":{"avgcount":28,"sum":0.007213381,"avgtime":0.000257620},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000279,"avgtime":0.000000139},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.003600788,"avgtime":0.000066681}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000108880,"avgtime":0.000027220}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":2,"traverse_hit":2,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":1,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":3,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198220,"rdpos":4195244,"jlat":{"avgcount":3,"sum":0.006218651,"avgtime":0.002072883},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":48532,"heap":280825856},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":772,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229616,"buffer_anon_items":71,"buffer_meta_bytes":88,"buffer_meta_items":1,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5736,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56376,"mds_co_items":248,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":53,"op_send_bytes":3974,"op_resend":0,"op_reply":53,"op_latency":{"avgcount":53,"sum":0.060999538,"avgtime":0.001150934},"op_inflight":0,"oplen_avg":{"avgcount":53,"sum":107},"op":53,"op_r":36,"op_w":17,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":53,"map_full":0,"map_inc":2,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":11,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":240,"get_sum":186862,"get_or_fail_fail":0,"get_or_fail_success":240,"take":0,"take_sum":0,"put":240,"put_sum":186862,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":53,"take_sum":4906,"put":53,"put_sum":4906,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":53,"take_sum":53,"put":53,"put_sum":53,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55e8d05a6d20":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2720,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2720,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-10T22:40:52.864 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 143, "avgtime": 1.89e-07, "sum": 2.7074e-05 }, "msgr_recv_bytes": 183509, "msgr_recv_encrypted_bytes": 129680, "msgr_recv_messages": 191, "msgr_running_fast_dispatch_time": 0.00753249, "msgr_running_recv_time": 0.014528693, "msgr_running_send_time": 0.011879793, "msgr_running_total_time": 0.0339324, "msgr_send_bytes": 29277, "msgr_send_encrypted_bytes": 25504, "msgr_send_messages": 64, "msgr_send_messages_queue_lat": { "avgcount": 64, "avgtime": 0.000137715, "sum": 0.008813789 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 13068, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 25, "msgr_running_fast_dispatch_time": 0.000532521, "msgr_running_recv_time": 0.002727653, "msgr_running_send_time": 0.002612589, "msgr_running_total_time": 0.006321962, "msgr_send_bytes": 147947, "msgr_send_encrypted_bytes": 136176, "msgr_send_messages": 52, "msgr_send_messages_queue_lat": { "avgcount": 51, "avgtime": 7.5025e-05, "sum": 0.003826316 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.39e-07, "sum": 2.79e-07 }, "msgr_recv_bytes": 16401, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 28, "msgr_running_fast_dispatch_time": 0.000586668, "msgr_running_recv_time": 0.002714958, "msgr_running_send_time": 0.001036415, "msgr_running_total_time": 0.004646581, "msgr_send_bytes": 35106, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 28, "avgtime": 0.00025762, "sum": 0.007213381 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 6.6681e-05, "sum": 0.003600788 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.722e-05, "sum": 0.00010888 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "request": 3, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 2, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 2, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 6, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.002072883, "sum": 0.006218651 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4198220 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 280825856, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48532 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 772, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229616, "buffer_anon_items": 71, "buffer_meta_bytes": 88, "buffer_meta_items": 1, "mds_co_bytes": 56376, "mds_co_items": 248, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5736, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 53, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 11, "op": 53, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 53, "avgtime": 0.001150934, "sum": 0.060999538 }, "op_pg": 0, "op_r": 36, "op_reply": 53, "op_resend": 0, "op_rmw": 0, "op_send": 53, "op_send_bytes": 3974, "op_w": 17, "oplen_avg": { "avgcount": 53, "sum": 107 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 240, "get_or_fail_fail": 0, "get_or_fail_success": 240, "get_started": 0, "get_sum": 186862, "max": 104857600, "put": 240, "put_sum": 186862, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 53, "put_sum": 4906, "take": 53, "take_sum": 4906, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 53, "put_sum": 53, "take": 53, "take_sum": 53, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55e8d05a6d20": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2720, "max": 3758096384, "put": 3, "put_sum": 2720, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-10T22:40:52.864 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_1:2 2024-09-10T22:40:52.864 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.0/omap_counter_test_file", \'w\') as f:\n f.write("content")\n f.flush()\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi066') 2024-09-10T22:40:52.866 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-10T22:40:52.911 INFO:teuthology.orchestra.run.smithi066.stderr:open_background: keeping file open 2024-09-10T22:40:52.914 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:40:52.920 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:53.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.063+0000 7fcb2b233640 1 -- 172.21.15.66:0/3508073341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb2406ed20 msgr2=0x7fcb240701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:53.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.063+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3508073341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb2406ed20 0x7fcb240701c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcb18005fd0 tx=0x7fcb1802a670 comp rx=0 tx=0).stop 2024-09-10T22:40:53.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.063+0000 7fcb2b233640 1 -- 172.21.15.66:0/3508073341 shutdown_connections 2024-09-10T22:40:53.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.064+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3508073341 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb2406ed20 0x7fcb240701c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:53.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.064+0000 7fcb2b233640 1 -- 172.21.15.66:0/3508073341 >> 172.21.15.66:0/3508073341 conn(0x7fcb240b1950 msgr2=0x7fcb240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:53.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.064+0000 7fcb2b233640 1 -- 172.21.15.66:0/3508073341 shutdown_connections 2024-09-10T22:40:53.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.064+0000 7fcb2b233640 1 -- 172.21.15.66:0/3508073341 wait complete. 2024-09-10T22:40:53.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.065+0000 7fcb2b233640 1 Processor -- start 2024-09-10T22:40:53.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.065+0000 7fcb2b233640 1 -- start start 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb2b233640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb2406ed20 0x7fcb241597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb2b233640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb24072880 0x7fcb241f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb2b233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb2415ffe0 0x7fcb241488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb2b233640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24074ae0 con 0x7fcb24072880 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb2b233640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24074960 con 0x7fcb2415ffe0 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb2b233640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fcb240cccf0 con 0x7fcb2406ed20 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb24072880 0x7fcb241f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb24072880 0x7fcb241f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58020/0 (socket says 172.21.15.66:58020) 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 -- 172.21.15.66:0/3853612197 learned_addr learned my addr 172.21.15.66:0/3853612197 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb297d0640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb2415ffe0 0x7fcb241488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb28fcf640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb2406ed20 0x7fcb241597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:53.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 -- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb2406ed20 msgr2=0x7fcb241597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb2406ed20 0x7fcb241597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 -- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb2415ffe0 msgr2=0x7fcb241488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb2415ffe0 0x7fcb241488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 -- 172.21.15.66:0/3853612197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb241f70a0 con 0x7fcb24072880 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb23fff640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb24072880 0x7fcb241f6b60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c450 tx=0x7fcb1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb217fa640 1 -- 172.21.15.66:0/3853612197 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100052d0 con 0x7fcb24072880 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb217fa640 1 -- 172.21.15.66:0/3853612197 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb10005470 con 0x7fcb24072880 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb217fa640 1 -- 172.21.15.66:0/3853612197 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10012870 con 0x7fcb24072880 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.066+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb24158710 con 0x7fcb24072880 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.067+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb24158cd0 con 0x7fcb24072880 2024-09-10T22:40:53.071 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.067+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb240cd1a0 con 0x7fcb24072880 2024-09-10T22:40:53.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.068+0000 7fcb217fa640 1 -- 172.21.15.66:0/3853612197 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcb100035f0 con 0x7fcb24072880 2024-09-10T22:40:53.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.069+0000 7fcb217fa640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcb08077f70 0x7fcb0807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:53.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.069+0000 7fcb217fa640 1 -- 172.21.15.66:0/3853612197 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7fcb10099940 con 0x7fcb24072880 2024-09-10T22:40:53.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.069+0000 7fcb28fcf640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcb08077f70 0x7fcb0807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:53.072 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.069+0000 7fcb28fcf640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcb08077f70 0x7fcb0807a430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcb18033000 tx=0x7fcb18004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:53.075 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.074+0000 7fcb217fa640 1 -- 172.21.15.66:0/3853612197 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcb241f70a0 con 0x7fcb24072880 2024-09-10T22:40:53.295 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.293+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb240b2010 con 0x7fcb24072880 2024-09-10T22:40:53.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.294+0000 7fcb217fa640 1 -- 172.21.15.66:0/3853612197 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7fcb100623b0 con 0x7fcb24072880 2024-09-10T22:40:53.296 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:53.296 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:53.296 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcb08077f70 msgr2=0x7fcb0807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcb08077f70 0x7fcb0807a430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcb18033000 tx=0x7fcb18004730 comp rx=0 tx=0).stop 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb24072880 msgr2=0x7fcb241f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb24072880 0x7fcb241f6b60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c450 tx=0x7fcb1000c910 comp rx=0 tx=0).stop 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 shutdown_connections 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fcb08077f70 0x7fcb0807a430 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb2415ffe0 0x7fcb241488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcb24072880 0x7fcb241f6b60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:53.298 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 --2- 172.21.15.66:0/3853612197 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fcb2406ed20 0x7fcb241597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:53.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 >> 172.21.15.66:0/3853612197 conn(0x7fcb240b1950 msgr2=0x7fcb240ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:53.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 shutdown_connections 2024-09-10T22:40:53.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:53.296+0000 7fcb2b233640 1 -- 172.21.15.66:0/3853612197 wait complete. 2024-09-10T22:40:53.309 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-10T22:40:53.429 INFO:teuthology.orchestra.run.smithi066.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":192,"msgr_send_messages":64,"msgr_recv_bytes":183927,"msgr_send_bytes":29277,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.034055487,"msgr_running_send_time":0.011921843,"msgr_running_recv_time":0.014558334,"msgr_running_fast_dispatch_time":0.007579944,"msgr_send_messages_queue_lat":{"avgcount":64,"sum":0.008813789,"avgtime":0.000137715},"msgr_handle_ack_lat":{"avgcount":144,"sum":0.000027225,"avgtime":0.000000189},"msgr_recv_encrypted_bytes":129680,"msgr_send_encrypted_bytes":25504},"AsyncMessenger::Worker-1":{"msgr_recv_messages":25,"msgr_send_messages":52,"msgr_recv_bytes":13068,"msgr_send_bytes":147947,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006321962,"msgr_running_send_time":0.002612589,"msgr_running_recv_time":0.002727653,"msgr_running_fast_dispatch_time":0.000532521,"msgr_send_messages_queue_lat":{"avgcount":51,"sum":0.003826316,"avgtime":0.000075025},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":136176},"AsyncMessenger::Worker-2":{"msgr_recv_messages":28,"msgr_send_messages":30,"msgr_recv_bytes":16401,"msgr_send_bytes":35106,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.004646581,"msgr_running_send_time":0.001036415,"msgr_running_recv_time":0.002714958,"msgr_running_fast_dispatch_time":0.000586668,"msgr_send_messages_queue_lat":{"avgcount":28,"sum":0.007213381,"avgtime":0.000257620},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000279,"avgtime":0.000000139},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.003600788,"avgtime":0.000066681}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000108880,"avgtime":0.000027220}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":2,"traverse_hit":2,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":1,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":3,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198220,"rdpos":4195244,"jlat":{"avgcount":3,"sum":0.006218651,"avgtime":0.002072883},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":48532,"heap":280825856},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":7,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":772,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229616,"buffer_anon_items":71,"buffer_meta_bytes":88,"buffer_meta_items":1,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5736,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56376,"mds_co_items":248,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":53,"op_send_bytes":3974,"op_resend":0,"op_reply":53,"op_latency":{"avgcount":53,"sum":0.060999538,"avgtime":0.001150934},"op_inflight":0,"oplen_avg":{"avgcount":53,"sum":107},"op":53,"op_r":36,"op_w":17,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":53,"map_full":0,"map_inc":2,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":11,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":241,"get_sum":187190,"get_or_fail_fail":0,"get_or_fail_success":241,"take":0,"take_sum":0,"put":241,"put_sum":187190,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":53,"take_sum":4906,"put":53,"put_sum":4906,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":53,"take_sum":53,"put":53,"put_sum":53,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55e8d05a6d20":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2720,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2720,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-10T22:40:53.430 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 144, "avgtime": 1.89e-07, "sum": 2.7225e-05 }, "msgr_recv_bytes": 183927, "msgr_recv_encrypted_bytes": 129680, "msgr_recv_messages": 192, "msgr_running_fast_dispatch_time": 0.007579944, "msgr_running_recv_time": 0.014558334, "msgr_running_send_time": 0.011921843, "msgr_running_total_time": 0.034055487, "msgr_send_bytes": 29277, "msgr_send_encrypted_bytes": 25504, "msgr_send_messages": 64, "msgr_send_messages_queue_lat": { "avgcount": 64, "avgtime": 0.000137715, "sum": 0.008813789 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 13068, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 25, "msgr_running_fast_dispatch_time": 0.000532521, "msgr_running_recv_time": 0.002727653, "msgr_running_send_time": 0.002612589, "msgr_running_total_time": 0.006321962, "msgr_send_bytes": 147947, "msgr_send_encrypted_bytes": 136176, "msgr_send_messages": 52, "msgr_send_messages_queue_lat": { "avgcount": 51, "avgtime": 7.5025e-05, "sum": 0.003826316 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.39e-07, "sum": 2.79e-07 }, "msgr_recv_bytes": 16401, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 28, "msgr_running_fast_dispatch_time": 0.000586668, "msgr_running_recv_time": 0.002714958, "msgr_running_send_time": 0.001036415, "msgr_running_total_time": 0.004646581, "msgr_send_bytes": 35106, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 28, "avgtime": 0.00025762, "sum": 0.007213381 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 6.6681e-05, "sum": 0.003600788 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.722e-05, "sum": 0.00010888 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "request": 3, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 2, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 2, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 6, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.002072883, "sum": 0.006218651 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4198220 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 280825856, "ino": 14, "ino+": 14, "ino-": 0, "rss": 48532 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 7, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 772, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229616, "buffer_anon_items": 71, "buffer_meta_bytes": 88, "buffer_meta_items": 1, "mds_co_bytes": 56376, "mds_co_items": 248, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5736, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 53, "map_full": 0, "map_inc": 2, "omap_del": 3, "omap_rd": 48, "omap_wr": 11, "op": 53, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 53, "avgtime": 0.001150934, "sum": 0.060999538 }, "op_pg": 0, "op_r": 36, "op_reply": 53, "op_resend": 0, "op_rmw": 0, "op_send": 53, "op_send_bytes": 3974, "op_w": 17, "oplen_avg": { "avgcount": 53, "sum": 107 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 241, "get_or_fail_fail": 0, "get_or_fail_success": 241, "get_started": 0, "get_sum": 187190, "max": 104857600, "put": 241, "put_sum": 187190, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 53, "put_sum": 4906, "take": 53, "take_sum": 4906, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 53, "put_sum": 53, "take": 53, "take_sum": 53, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55e8d05a6d20": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2720, "max": 3758096384, "put": 3, "put_sum": 2720, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-10T22:40:53.430 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-10T22:40:58.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 -- 172.21.15.66:0/3447651297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb40cc1c0 msgr2=0x7efeb40747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3447651297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb40cc1c0 0x7efeb40747a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efeb4073e30 tx=0x7efea802a560 comp rx=0 tx=0).stop 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 -- 172.21.15.66:0/3447651297 shutdown_connections 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3447651297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb40cc1c0 0x7efeb40747a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3447651297 >> v2:172.21.15.94:3300/0 conn(0x7efeb4158c90 0x7efeb415c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 -- 172.21.15.66:0/3447651297 >> 172.21.15.66:0/3447651297 conn(0x7efeb40731e0 msgr2=0x7efeb4074090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 -- 172.21.15.66:0/3447651297 shutdown_connections 2024-09-10T22:40:58.551 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.549+0000 7efeb9f64640 1 -- 172.21.15.66:0/3447651297 wait complete. 2024-09-10T22:40:58.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.551+0000 7efeb9f64640 1 Processor -- start 2024-09-10T22:40:58.552 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.551+0000 7efeb9f64640 1 -- start start 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb9f64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb40cc1c0 0x7efeb4157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb9f64640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efeb40ccb90 0x7efeb40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb9f64640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb4158c90 0x7efeb40c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb9f64640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7efeb40cd460 con 0x7efeb4158c90 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb9f64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efeb40cd2e0 con 0x7efeb40cc1c0 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb9f64640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7efeb415f170 con 0x7efeb40ccb90 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb4158c90 0x7efeb40c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb4158c90 0x7efeb40c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39104/0 (socket says 172.21.15.66:39104) 2024-09-10T22:40:58.554 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 -- 172.21.15.66:0/3761810166 learned_addr learned my addr 172.21.15.66:0/3761810166 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb37fe640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb40cc1c0 0x7efeb4157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 -- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efeb40ccb90 msgr2=0x7efeb40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efeb40ccb90 0x7efeb40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 -- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb40cc1c0 msgr2=0x7efeb4157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb40cc1c0 0x7efeb4157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 -- 172.21.15.66:0/3761810166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb41f4df0 con 0x7efeb4158c90 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb3fff640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb4158c90 0x7efeb40c57f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c7b0 tx=0x7efeac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efe9bfff640 1 -- 172.21.15.66:0/3761810166 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac004b60 con 0x7efeb4158c90 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efe9bfff640 1 -- 172.21.15.66:0/3761810166 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeac004df0 con 0x7efeb4158c90 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb40b5f90 con 0x7efeb4158c90 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.552+0000 7efe9bfff640 1 -- 172.21.15.66:0/3761810166 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac00aa90 con 0x7efeb4158c90 2024-09-10T22:40:58.555 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.553+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb40b5cb0 con 0x7efeb4158c90 2024-09-10T22:40:58.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.553+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb415f620 con 0x7efeb4158c90 2024-09-10T22:40:58.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.554+0000 7efe9bfff640 1 -- 172.21.15.66:0/3761810166 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efeac00ac30 con 0x7efeb4158c90 2024-09-10T22:40:58.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.555+0000 7efe9bfff640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efea0077f70 0x7efea007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:58.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.555+0000 7efe9bfff640 1 -- 172.21.15.66:0/3761810166 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7efeac09a020 con 0x7efeb4158c90 2024-09-10T22:40:58.556 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.555+0000 7efeb37fe640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efea0077f70 0x7efea007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:58.557 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.556+0000 7efeb37fe640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efea0077f70 0x7efea007a430 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efea400b9a0 tx=0x7efea4005c20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:58.560 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.559+0000 7efe9bfff640 1 -- 172.21.15.66:0/3761810166 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7efeac020070 con 0x7efeb4158c90 2024-09-10T22:40:58.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.780+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeb40cd870 con 0x7efeb4158c90 2024-09-10T22:40:58.781 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.780+0000 7efe9bfff640 1 -- 172.21.15.66:0/3761810166 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7efeac062b20 con 0x7efeb4158c90 2024-09-10T22:40:58.782 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:58.782 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:58.782 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.782+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efea0077f70 msgr2=0x7efea007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.782+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efea0077f70 0x7efea007a430 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efea400b9a0 tx=0x7efea4005c20 comp rx=0 tx=0).stop 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.782+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb4158c90 msgr2=0x7efeb40c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.782+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb4158c90 0x7efeb40c57f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c7b0 tx=0x7efeac00cc70 comp rx=0 tx=0).stop 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 shutdown_connections 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7efea0077f70 0x7efea007a430 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efeb4158c90 0x7efeb40c57f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7efeb40ccb90 0x7efeb40d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 --2- 172.21.15.66:0/3761810166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efeb40cc1c0 0x7efeb4157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:58.784 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 >> 172.21.15.66:0/3761810166 conn(0x7efeb40731e0 msgr2=0x7efeb415a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:58.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 shutdown_connections 2024-09-10T22:40:58.785 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:58.783+0000 7efeb9f64640 1 -- 172.21.15.66:0/3761810166 wait complete. 2024-09-10T22:40:58.795 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-10T22:40:58.914 INFO:teuthology.orchestra.run.smithi066.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":200,"msgr_send_messages":67,"msgr_recv_bytes":186757,"msgr_send_bytes":30385,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.035183885,"msgr_running_send_time":0.012403111,"msgr_running_recv_time":0.014926171,"msgr_running_fast_dispatch_time":0.007881342,"msgr_send_messages_queue_lat":{"avgcount":67,"sum":0.008907963,"avgtime":0.000132954},"msgr_handle_ack_lat":{"avgcount":153,"sum":0.000028641,"avgtime":0.000000187},"msgr_recv_encrypted_bytes":129952,"msgr_send_encrypted_bytes":26144},"AsyncMessenger::Worker-1":{"msgr_recv_messages":26,"msgr_send_messages":55,"msgr_recv_bytes":13314,"msgr_send_bytes":153685,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006565219,"msgr_running_send_time":0.002775222,"msgr_running_recv_time":0.002756925,"msgr_running_fast_dispatch_time":0.000560753,"msgr_send_messages_queue_lat":{"avgcount":54,"sum":0.003923802,"avgtime":0.000072663},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":139856},"AsyncMessenger::Worker-2":{"msgr_recv_messages":31,"msgr_send_messages":33,"msgr_recv_bytes":17235,"msgr_send_bytes":36360,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.005023636,"msgr_running_send_time":0.001224750,"msgr_running_recv_time":0.002797984,"msgr_running_fast_dispatch_time":0.000668200,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.007361909,"avgtime":0.000237480},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000279,"avgtime":0.000000139},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":58,"sum":0.003837017,"avgtime":0.000066155}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000108880,"avgtime":0.000027220}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":2,"traverse_hit":2,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":1,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":2,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":4,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199960,"rdpos":4195244,"jlat":{"avgcount":4,"sum":0.008114993,"avgtime":0.002028748},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":49044,"heap":280825856},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":8,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":777,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229616,"buffer_anon_items":71,"buffer_meta_bytes":88,"buffer_meta_items":1,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5736,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56352,"mds_co_items":252,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":57,"op_send_bytes":5926,"op_resend":0,"op_reply":57,"op_latency":{"avgcount":57,"sum":0.067999483,"avgtime":0.001192973},"op_inflight":0,"oplen_avg":{"avgcount":57,"sum":113},"op":57,"op_r":36,"op_w":21,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":53,"map_full":0,"map_inc":2,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":253,"get_sum":189968,"get_or_fail_fail":0,"get_or_fail_success":253,"take":0,"take_sum":0,"put":253,"put_sum":189968,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":6858,"put":57,"put_sum":6858,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":57,"put":57,"put_sum":57,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55e8d05a6d20":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4460,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4460,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-10T22:40:58.915 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 153, "avgtime": 1.87e-07, "sum": 2.8641e-05 }, "msgr_recv_bytes": 186757, "msgr_recv_encrypted_bytes": 129952, "msgr_recv_messages": 200, "msgr_running_fast_dispatch_time": 0.007881342, "msgr_running_recv_time": 0.014926171, "msgr_running_send_time": 0.012403111, "msgr_running_total_time": 0.035183885, "msgr_send_bytes": 30385, "msgr_send_encrypted_bytes": 26144, "msgr_send_messages": 67, "msgr_send_messages_queue_lat": { "avgcount": 67, "avgtime": 0.000132954, "sum": 0.008907963 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 13314, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 26, "msgr_running_fast_dispatch_time": 0.000560753, "msgr_running_recv_time": 0.002756925, "msgr_running_send_time": 0.002775222, "msgr_running_total_time": 0.006565219, "msgr_send_bytes": 153685, "msgr_send_encrypted_bytes": 139856, "msgr_send_messages": 55, "msgr_send_messages_queue_lat": { "avgcount": 54, "avgtime": 7.2663e-05, "sum": 0.003923802 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.39e-07, "sum": 2.79e-07 }, "msgr_recv_bytes": 17235, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 31, "msgr_running_fast_dispatch_time": 0.0006682, "msgr_running_recv_time": 0.002797984, "msgr_running_send_time": 0.00122475, "msgr_running_total_time": 0.005023636, "msgr_send_bytes": 36360, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.00023748, "sum": 0.007361909 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 58, "avgtime": 6.6155e-05, "sum": 0.003837017 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.722e-05, "sum": 0.00010888 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 2, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "request": 3, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 2, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 2, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 7, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.002028748, "sum": 0.008114993 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4199960 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 280825856, "ino": 14, "ino+": 14, "ino-": 0, "rss": 49044 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 8, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 777, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229616, "buffer_anon_items": 71, "buffer_meta_bytes": 88, "buffer_meta_items": 1, "mds_co_bytes": 56352, "mds_co_items": 252, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5736, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 53, "map_full": 0, "map_inc": 2, "omap_del": 4, "omap_rd": 48, "omap_wr": 15, "op": 57, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 57, "avgtime": 0.001192973, "sum": 0.067999483 }, "op_pg": 0, "op_r": 36, "op_reply": 57, "op_resend": 0, "op_rmw": 0, "op_send": 57, "op_send_bytes": 5926, "op_w": 21, "oplen_avg": { "avgcount": 57, "sum": 113 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 253, "get_or_fail_fail": 0, "get_or_fail_success": 253, "get_started": 0, "get_sum": 189968, "max": 104857600, "put": 253, "put_sum": 189968, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 57, "put_sum": 6858, "take": 57, "take_sum": 6858, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 57, "put_sum": 57, "take": 57, "take_sum": 57, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55e8d05a6d20": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4460, "max": 3758096384, "put": 4, "put_sum": 4460, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-10T22:40:58.915 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T22:40:58.915 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:59.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 -- 172.21.15.66:0/3330656587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 msgr2=0x7f7f140c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 --2- 172.21.15.66:0/3330656587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f140c6590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f140ce460 tx=0x7f7f0402a5f0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 -- 172.21.15.66:0/3330656587 shutdown_connections 2024-09-10T22:40:59.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 --2- 172.21.15.66:0/3330656587 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f140c6590 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 --2- 172.21.15.66:0/3330656587 >> v2:172.21.15.94:3300/0 conn(0x7f7f140c56e0 0x7f7f14073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 -- 172.21.15.66:0/3330656587 >> 172.21.15.66:0/3330656587 conn(0x7f7f14071380 msgr2=0x7f7f1406cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 -- 172.21.15.66:0/3330656587 shutdown_connections 2024-09-10T22:40:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.020+0000 7f7f19860640 1 -- 172.21.15.66:0/3330656587 wait complete. 2024-09-10T22:40:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.021+0000 7f7f19860640 1 Processor -- start 2024-09-10T22:40:59.022 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.021+0000 7f7f19860640 1 -- start start 2024-09-10T22:40:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f19860640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f140b2de0 0x7f7f140cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f19860640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f140c56e0 0x7f7f1415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f19860640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f1415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f19860640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1415c680 con 0x7f7f140c6160 2024-09-10T22:40:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f19860640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1415c500 con 0x7f7f140b2de0 2024-09-10T22:40:59.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f19860640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f7f1415c800 con 0x7f7f140c56e0 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f1415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f1415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39120/0 (socket says 172.21.15.66:39120) 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 -- 172.21.15.66:0/4262951534 learned_addr learned my addr 172.21.15.66:0/4262951534 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f12ffd640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f140b2de0 0x7f7f140cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f127fc640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f140c56e0 0x7f7f1415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 -- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f140c56e0 msgr2=0x7f7f1415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f140c56e0 0x7f7f1415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 -- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f140b2de0 msgr2=0x7f7f140cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f140b2de0 0x7f7f140cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.022+0000 7f7f137fe640 1 -- 172.21.15.66:0/4262951534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f140c85b0 con 0x7f7f140c6160 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.023+0000 7f7f137fe640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f1415e130 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7f040099c0 tx=0x7f7f04003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:59.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.023+0000 7f7f0bfff640 1 -- 172.21.15.66:0/4262951534 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0403f070 con 0x7f7f140c6160 2024-09-10T22:40:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.023+0000 7f7f0bfff640 1 -- 172.21.15.66:0/4262951534 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f040047a0 con 0x7f7f140c6160 2024-09-10T22:40:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.023+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f140c8890 con 0x7f7f140c6160 2024-09-10T22:40:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.023+0000 7f7f0bfff640 1 -- 172.21.15.66:0/4262951534 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04035030 con 0x7f7f140c6160 2024-09-10T22:40:59.026 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.023+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f140c8e50 con 0x7f7f140c6160 2024-09-10T22:40:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.024+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f14073110 con 0x7f7f140c6160 2024-09-10T22:40:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.025+0000 7f7f0bfff640 1 -- 172.21.15.66:0/4262951534 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7f04004bf0 con 0x7f7f140c6160 2024-09-10T22:40:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.025+0000 7f7f0bfff640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ef4077f70 0x7f7ef407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.025+0000 7f7f0bfff640 1 -- 172.21.15.66:0/4262951534 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f7f040badc0 con 0x7f7f140c6160 2024-09-10T22:40:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.025+0000 7f7f12ffd640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ef4077f70 0x7f7ef407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.026+0000 7f7f12ffd640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ef4077f70 0x7f7ef407a430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c001e20 tx=0x7f7f0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:59.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.029+0000 7f7f0bfff640 1 -- 172.21.15.66:0/4262951534 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7f04083270 con 0x7f7f140c6160 2024-09-10T22:40:59.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.253+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f14075ad0 con 0x7f7f140c6160 2024-09-10T22:40:59.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.254+0000 7f7f0bfff640 1 -- 172.21.15.66:0/4262951534 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f7f04087ca0 con 0x7f7f140c6160 2024-09-10T22:40:59.255 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:59.255 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:59.255 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:59.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ef4077f70 msgr2=0x7f7ef407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ef4077f70 0x7f7ef407a430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c001e20 tx=0x7f7f0c009040 comp rx=0 tx=0).stop 2024-09-10T22:40:59.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 msgr2=0x7f7f1415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f1415e130 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7f040099c0 tx=0x7f7f04003960 comp rx=0 tx=0).stop 2024-09-10T22:40:59.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 shutdown_connections 2024-09-10T22:40:59.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7ef4077f70 0x7f7ef407a430 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7f140c6160 0x7f7f1415e130 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f7f140c56e0 0x7f7f1415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 --2- 172.21.15.66:0/4262951534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f140b2de0 0x7f7f140cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 >> 172.21.15.66:0/4262951534 conn(0x7f7f14071380 msgr2=0x7f7f140cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 shutdown_connections 2024-09-10T22:40:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.256+0000 7f7f19860640 1 -- 172.21.15.66:0/4262951534 wait complete. 2024-09-10T22:40:59.268 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-10T22:40:59.387 INFO:teuthology.orchestra.run.smithi066.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":200,"msgr_send_messages":67,"msgr_recv_bytes":186757,"msgr_send_bytes":30385,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.035183885,"msgr_running_send_time":0.012403111,"msgr_running_recv_time":0.014926171,"msgr_running_fast_dispatch_time":0.007881342,"msgr_send_messages_queue_lat":{"avgcount":67,"sum":0.008907963,"avgtime":0.000132954},"msgr_handle_ack_lat":{"avgcount":153,"sum":0.000028641,"avgtime":0.000000187},"msgr_recv_encrypted_bytes":129952,"msgr_send_encrypted_bytes":26144},"AsyncMessenger::Worker-1":{"msgr_recv_messages":26,"msgr_send_messages":55,"msgr_recv_bytes":13314,"msgr_send_bytes":153685,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006565219,"msgr_running_send_time":0.002775222,"msgr_running_recv_time":0.002756925,"msgr_running_fast_dispatch_time":0.000560753,"msgr_send_messages_queue_lat":{"avgcount":54,"sum":0.003923802,"avgtime":0.000072663},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":139856},"AsyncMessenger::Worker-2":{"msgr_recv_messages":31,"msgr_send_messages":33,"msgr_recv_bytes":17235,"msgr_send_bytes":36360,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.005023636,"msgr_running_send_time":0.001224750,"msgr_running_recv_time":0.002797984,"msgr_running_fast_dispatch_time":0.000668200,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.007361909,"avgtime":0.000237480},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000279,"avgtime":0.000000139},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":58,"sum":0.003837017,"avgtime":0.000066155}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000108880,"avgtime":0.000027220}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":2,"traverse_hit":2,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":1,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":2,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":4,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199960,"rdpos":4195244,"jlat":{"avgcount":4,"sum":0.008114993,"avgtime":0.002028748},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":49044,"heap":280825856},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":8,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":777,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229616,"buffer_anon_items":71,"buffer_meta_bytes":88,"buffer_meta_items":1,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5736,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56352,"mds_co_items":252,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":57,"op_send_bytes":5926,"op_resend":0,"op_reply":57,"op_latency":{"avgcount":57,"sum":0.067999483,"avgtime":0.001192973},"op_inflight":0,"oplen_avg":{"avgcount":57,"sum":113},"op":57,"op_r":36,"op_w":21,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":53,"map_full":0,"map_inc":2,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":253,"get_sum":189968,"get_or_fail_fail":0,"get_or_fail_success":253,"take":0,"take_sum":0,"put":253,"put_sum":189968,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":6858,"put":57,"put_sum":6858,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":57,"put":57,"put_sum":57,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55e8d05a6d20":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4460,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4460,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-10T22:40:59.388 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 153, "avgtime": 1.87e-07, "sum": 2.8641e-05 }, "msgr_recv_bytes": 186757, "msgr_recv_encrypted_bytes": 129952, "msgr_recv_messages": 200, "msgr_running_fast_dispatch_time": 0.007881342, "msgr_running_recv_time": 0.014926171, "msgr_running_send_time": 0.012403111, "msgr_running_total_time": 0.035183885, "msgr_send_bytes": 30385, "msgr_send_encrypted_bytes": 26144, "msgr_send_messages": 67, "msgr_send_messages_queue_lat": { "avgcount": 67, "avgtime": 0.000132954, "sum": 0.008907963 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 13314, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 26, "msgr_running_fast_dispatch_time": 0.000560753, "msgr_running_recv_time": 0.002756925, "msgr_running_send_time": 0.002775222, "msgr_running_total_time": 0.006565219, "msgr_send_bytes": 153685, "msgr_send_encrypted_bytes": 139856, "msgr_send_messages": 55, "msgr_send_messages_queue_lat": { "avgcount": 54, "avgtime": 7.2663e-05, "sum": 0.003923802 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.39e-07, "sum": 2.79e-07 }, "msgr_recv_bytes": 17235, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 31, "msgr_running_fast_dispatch_time": 0.0006682, "msgr_running_recv_time": 0.002797984, "msgr_running_send_time": 0.00122475, "msgr_running_total_time": 0.005023636, "msgr_send_bytes": 36360, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.00023748, "sum": 0.007361909 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 58, "avgtime": 6.6155e-05, "sum": 0.003837017 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.722e-05, "sum": 0.00010888 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 2, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "request": 3, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 2, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 2, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 7, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.002028748, "sum": 0.008114993 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4199960 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 280825856, "ino": 14, "ino+": 14, "ino-": 0, "rss": 49044 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 8, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 777, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229616, "buffer_anon_items": 71, "buffer_meta_bytes": 88, "buffer_meta_items": 1, "mds_co_bytes": 56352, "mds_co_items": 252, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5736, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 53, "map_full": 0, "map_inc": 2, "omap_del": 4, "omap_rd": 48, "omap_wr": 15, "op": 57, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 57, "avgtime": 0.001192973, "sum": 0.067999483 }, "op_pg": 0, "op_r": 36, "op_reply": 57, "op_resend": 0, "op_rmw": 0, "op_send": 57, "op_send_bytes": 5926, "op_w": 21, "oplen_avg": { "avgcount": 57, "sum": 113 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 253, "get_or_fail_fail": 0, "get_or_fail_success": 253, "get_started": 0, "get_sum": 189968, "max": 104857600, "put": 253, "put_sum": 189968, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 57, "put_sum": 6858, "take": 57, "take_sum": 6858, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 57, "put_sum": 57, "take": 57, "take_sum": 57, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55e8d05a6d20": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4460, "max": 3758096384, "put": 4, "put_sum": 4460, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-10T22:40:59.388 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T22:40:59.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:40:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.492+0000 7f836d016640 1 -- 172.21.15.66:0/3651150540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680ccb90 msgr2=0x7f83680cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.492+0000 7f836d016640 1 --2- 172.21.15.66:0/3651150540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680ccb90 0x7f83680cd040 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8360005fd0 tx=0x7f836002a8b0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.492+0000 7f836d016640 1 -- 172.21.15.66:0/3651150540 shutdown_connections 2024-09-10T22:40:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.492+0000 7f836d016640 1 --2- 172.21.15.66:0/3651150540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680ccb90 0x7f83680cd040 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.493+0000 7f836d016640 1 --2- 172.21.15.66:0/3651150540 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f83680cc1c0 0x7f836815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.493+0000 7f836d016640 1 -- 172.21.15.66:0/3651150540 >> 172.21.15.66:0/3651150540 conn(0x7f83680731e0 msgr2=0x7f8368074090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.493+0000 7f836d016640 1 -- 172.21.15.66:0/3651150540 shutdown_connections 2024-09-10T22:40:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.493+0000 7f836d016640 1 -- 172.21.15.66:0/3651150540 wait complete. 2024-09-10T22:40:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.494+0000 7f836d016640 1 Processor -- start 2024-09-10T22:40:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.494+0000 7f836d016640 1 -- start start 2024-09-10T22:40:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836d016640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680cc1c0 0x7f83681f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.497 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836d016640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83680ccb90 0x7f8368153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836d016640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8368158c90 0x7f83681544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836d016640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f83680ce430 con 0x7f83680ccb90 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836d016640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f83680ce2b0 con 0x7f83680cc1c0 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836d016640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f83680ce5b0 con 0x7f8368158c90 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83680ccb90 0x7f8368153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83680ccb90 0x7f8368153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39136/0 (socket says 172.21.15.66:39136) 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 -- 172.21.15.66:0/293879415 learned_addr learned my addr 172.21.15.66:0/293879415 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836759e640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8368158c90 0x7f83681544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f8366d9d640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680cc1c0 0x7f83681f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.498 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 -- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8368158c90 msgr2=0x7f83681544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8368158c90 0x7f83681544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 -- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680cc1c0 msgr2=0x7f83681f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680cc1c0 0x7f83681f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.495+0000 7f836659c640 1 -- 172.21.15.66:0/293879415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83680afe20 con 0x7f83680ccb90 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.496+0000 7f836659c640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83680ccb90 0x7f8368153f90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f835000e9c0 tx=0x7f835000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.496+0000 7f835f7fe640 1 -- 172.21.15.66:0/293879415 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835000cd80 con 0x7f83680ccb90 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.496+0000 7f835f7fe640 1 -- 172.21.15.66:0/293879415 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8350004020 con 0x7f83680ccb90 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.496+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83680b0100 con 0x7f83680ccb90 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.496+0000 7f835f7fe640 1 -- 172.21.15.66:0/293879415 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350004450 con 0x7f83680ccb90 2024-09-10T22:40:59.499 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.496+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83680b06c0 con 0x7f83680ccb90 2024-09-10T22:40:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.497+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83680cea60 con 0x7f83680ccb90 2024-09-10T22:40:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.497+0000 7f835f7fe640 1 -- 172.21.15.66:0/293879415 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f83500035f0 con 0x7f83680ccb90 2024-09-10T22:40:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.498+0000 7f835f7fe640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8348077f70 0x7f834807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:40:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.498+0000 7f835f7fe640 1 -- 172.21.15.66:0/293879415 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f835009a760 con 0x7f83680ccb90 2024-09-10T22:40:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.498+0000 7f8366d9d640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8348077f70 0x7f834807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:40:59.501 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.499+0000 7f8366d9d640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8348077f70 0x7f834807a430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8358001720 tx=0x7f8358007e90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:40:59.503 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.502+0000 7f835f7fe640 1 -- 172.21.15.66:0/293879415 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8350062b80 con 0x7f83680ccb90 2024-09-10T22:40:59.725 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.724+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83680cd870 con 0x7f83680ccb90 2024-09-10T22:40:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.725+0000 7f835f7fe640 1 -- 172.21.15.66:0/293879415 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f83500675b0 con 0x7f83680ccb90 2024-09-10T22:40:59.726 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:40:59.726 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:40:59.726 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8348077f70 msgr2=0x7f834807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8348077f70 0x7f834807a430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8358001720 tx=0x7f8358007e90 comp rx=0 tx=0).stop 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83680ccb90 msgr2=0x7f8368153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83680ccb90 0x7f8368153f90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f835000e9c0 tx=0x7f835000ee80 comp rx=0 tx=0).stop 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 shutdown_connections 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f8348077f70 0x7f834807a430 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f8368158c90 0x7f83681544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f83680ccb90 0x7f8368153f90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 --2- 172.21.15.66:0/293879415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f83680cc1c0 0x7f83681f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 >> 172.21.15.66:0/293879415 conn(0x7f83680731e0 msgr2=0x7f83680c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:40:59.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.727+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 shutdown_connections 2024-09-10T22:40:59.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:40:59.728+0000 7f836d016640 1 -- 172.21.15.66:0/293879415 wait complete. 2024-09-10T22:40:59.739 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-10T22:40:59.858 INFO:teuthology.orchestra.run.smithi066.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":202,"msgr_send_messages":68,"msgr_recv_bytes":187447,"msgr_send_bytes":31025,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.035388911,"msgr_running_send_time":0.012481804,"msgr_running_recv_time":0.014998908,"msgr_running_fast_dispatch_time":0.007938472,"msgr_send_messages_queue_lat":{"avgcount":68,"sum":0.008932815,"avgtime":0.000131364},"msgr_handle_ack_lat":{"avgcount":154,"sum":0.000028780,"avgtime":0.000000186},"msgr_recv_encrypted_bytes":130224,"msgr_send_encrypted_bytes":26784},"AsyncMessenger::Worker-1":{"msgr_recv_messages":26,"msgr_send_messages":55,"msgr_recv_bytes":13314,"msgr_send_bytes":153685,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.006565219,"msgr_running_send_time":0.002775222,"msgr_running_recv_time":0.002756925,"msgr_running_fast_dispatch_time":0.000560753,"msgr_send_messages_queue_lat":{"avgcount":54,"sum":0.003923802,"avgtime":0.000072663},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":139856},"AsyncMessenger::Worker-2":{"msgr_recv_messages":31,"msgr_send_messages":33,"msgr_recv_bytes":17235,"msgr_send_bytes":36360,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.005023636,"msgr_running_send_time":0.001224750,"msgr_running_recv_time":0.002797984,"msgr_running_fast_dispatch_time":0.000668200,"msgr_send_messages_queue_lat":{"avgcount":31,"sum":0.007361909,"avgtime":0.000237480},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000279,"avgtime":0.000000139},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":58,"sum":0.003837017,"avgtime":0.000066155}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000108880,"avgtime":0.000027220}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":2,"traverse_hit":2,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":1,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":2,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":4,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4199960,"rdpos":4195244,"jlat":{"avgcount":4,"sum":0.008114993,"avgtime":0.002028748},"replayed":3},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":49044,"heap":280825856},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":8,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.002814284,"avgtime":0.002814284},"req_getattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":777,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229616,"buffer_anon_items":71,"buffer_meta_bytes":88,"buffer_meta_items":1,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5736,"osdmap_items":94,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56352,"mds_co_items":252,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":57,"op_send_bytes":5926,"op_resend":0,"op_reply":57,"op_latency":{"avgcount":57,"sum":0.067999483,"avgtime":0.001192973},"op_inflight":0,"oplen_avg":{"avgcount":57,"sum":113},"op":57,"op_r":36,"op_w":21,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":53,"map_full":0,"map_inc":2,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":255,"get_sum":190426,"get_or_fail_fail":0,"get_or_fail_success":255,"take":0,"take_sum":0,"put":255,"put_sum":190426,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":6858,"put":57,"put_sum":6858,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":57,"put":57,"put_sum":57,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55e8d05a6d20":{"val":0,"max":3758096384,"get_started":0,"get":4,"get_sum":4460,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":4,"put_sum":4460,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":0}} 2024-09-10T22:40:59.859 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 154, "avgtime": 1.86e-07, "sum": 2.878e-05 }, "msgr_recv_bytes": 187447, "msgr_recv_encrypted_bytes": 130224, "msgr_recv_messages": 202, "msgr_running_fast_dispatch_time": 0.007938472, "msgr_running_recv_time": 0.014998908, "msgr_running_send_time": 0.012481804, "msgr_running_total_time": 0.035388911, "msgr_send_bytes": 31025, "msgr_send_encrypted_bytes": 26784, "msgr_send_messages": 68, "msgr_send_messages_queue_lat": { "avgcount": 68, "avgtime": 0.000131364, "sum": 0.008932815 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 13314, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 26, "msgr_running_fast_dispatch_time": 0.000560753, "msgr_running_recv_time": 0.002756925, "msgr_running_send_time": 0.002775222, "msgr_running_total_time": 0.006565219, "msgr_send_bytes": 153685, "msgr_send_encrypted_bytes": 139856, "msgr_send_messages": 55, "msgr_send_messages_queue_lat": { "avgcount": 54, "avgtime": 7.2663e-05, "sum": 0.003923802 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.39e-07, "sum": 2.79e-07 }, "msgr_recv_bytes": 17235, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 31, "msgr_running_fast_dispatch_time": 0.0006682, "msgr_running_recv_time": 0.002797984, "msgr_running_send_time": 0.00122475, "msgr_running_total_time": 0.005023636, "msgr_send_bytes": 36360, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 31, "avgtime": 0.00023748, "sum": 0.007361909 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 58, "avgtime": 6.6155e-05, "sum": 0.003837017 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 2.722e-05, "sum": 0.00010888 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 1, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 2, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "request": 3, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 2, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 2, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 7, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.002028748, "sum": 0.008114993 }, "rdpos": 4195244, "replayed": 3, "seg": 2, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4199960 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 280825856, "ino": 14, "ino+": 14, "ino-": 0, "rss": 49044 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 8, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.002814284, "sum": 0.002814284 }, "req_getattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 777, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229616, "buffer_anon_items": 71, "buffer_meta_bytes": 88, "buffer_meta_items": 1, "mds_co_bytes": 56352, "mds_co_items": 252, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5736, "osdmap_items": 94, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 53, "map_full": 0, "map_inc": 2, "omap_del": 4, "omap_rd": 48, "omap_wr": 15, "op": 57, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 57, "avgtime": 0.001192973, "sum": 0.067999483 }, "op_pg": 0, "op_r": 36, "op_reply": 57, "op_resend": 0, "op_rmw": 0, "op_send": 57, "op_send_bytes": 5926, "op_w": 21, "oplen_avg": { "avgcount": 57, "sum": 113 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 255, "get_or_fail_fail": 0, "get_or_fail_success": 255, "get_started": 0, "get_sum": 190426, "max": 104857600, "put": 255, "put_sum": 190426, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 57, "put_sum": 6858, "take": 57, "take_sum": 6858, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 57, "put_sum": 57, "take": 57, "take_sum": 57, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55e8d05a6d20": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4460, "max": 3758096384, "put": 4, "put_sum": 4460, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 0 } } 2024-09-10T22:40:59.859 INFO:tasks.cephfs.test_openfiletable:omap_total_removes:1 2024-09-10T22:40:59.859 INFO:tasks.cephfs.test_openfiletable:omap_total_kv_pairs:1 2024-09-10T22:40:59.860 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T22:40:59.860 DEBUG:teuthology.orchestra.run.smithi066:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:40:59.915 DEBUG:teuthology.orchestra.run.smithi094:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T22:40:59.961 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T22:40:59.961 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:40:59.961 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:40:59.961 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:40:59.977 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:40:59.978 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:00.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T22:41:00.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.180+0000 7ff826233640 1 -- 172.21.15.66:0/2620386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200b5be0 msgr2=0x7ff8200cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.180+0000 7ff826233640 1 --2- 172.21.15.66:0/2620386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200b5be0 0x7ff8200cd040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff82006d6a0 tx=0x7ff81002a5f0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.181+0000 7ff826233640 1 -- 172.21.15.66:0/2620386497 shutdown_connections 2024-09-10T22:41:00.182 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.181+0000 7ff826233640 1 --2- 172.21.15.66:0/2620386497 >> v2:172.21.15.94:3300/0 conn(0x7ff8200cd580 0x7ff8200b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.181+0000 7ff826233640 1 --2- 172.21.15.66:0/2620386497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200b5be0 0x7ff8200cd040 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.181+0000 7ff826233640 1 -- 172.21.15.66:0/2620386497 >> 172.21.15.66:0/2620386497 conn(0x7ff820069600 msgr2=0x7ff82006cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:00.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.181+0000 7ff826233640 1 -- 172.21.15.66:0/2620386497 shutdown_connections 2024-09-10T22:41:00.183 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.181+0000 7ff826233640 1 -- 172.21.15.66:0/2620386497 wait complete. 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.182+0000 7ff826233640 1 Processor -- start 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.182+0000 7ff826233640 1 -- start start 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff826233640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8200b5be0 0x7ff82015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff826233640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8200b5fc0 0x7ff82015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff826233640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200cd580 0x7ff820154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff826233640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ff82006f6a0 con 0x7ff8200cd580 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff826233640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff82006f520 con 0x7ff8200b5be0 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff826233640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ff82006f820 con 0x7ff8200b5fc0 2024-09-10T22:41:00.186 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200cd580 0x7ff820154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200cd580 0x7ff820154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39154/0 (socket says 172.21.15.66:39154) 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 -- 172.21.15.66:0/3770282051 learned_addr learned my addr 172.21.15.66:0/3770282051 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81f7fe640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8200b5be0 0x7ff82015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81effd640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8200b5fc0 0x7ff82015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 -- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8200b5fc0 msgr2=0x7ff82015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8200b5fc0 0x7ff82015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 -- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8200b5be0 msgr2=0x7ff82015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8200b5be0 0x7ff82015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 -- 172.21.15.66:0/3770282051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff820155270 con 0x7ff8200cd580 2024-09-10T22:41:00.187 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff81ffff640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200cd580 0x7ff820154540 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c7a0 tx=0x7ff81800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:00.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff807fff640 1 -- 172.21.15.66:0/3770282051 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818003b70 con 0x7ff8200cd580 2024-09-10T22:41:00.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff807fff640 1 -- 172.21.15.66:0/3770282051 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff818003d10 con 0x7ff8200cd580 2024-09-10T22:41:00.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8201450c0 con 0x7ff8200cd580 2024-09-10T22:41:00.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.183+0000 7ff807fff640 1 -- 172.21.15.66:0/3770282051 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81800aab0 con 0x7ff8200cd580 2024-09-10T22:41:00.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.184+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff820142940 con 0x7ff8200cd580 2024-09-10T22:41:00.188 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.184+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82006db00 con 0x7ff8200cd580 2024-09-10T22:41:00.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.185+0000 7ff807fff640 1 -- 172.21.15.66:0/3770282051 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff81800ac50 con 0x7ff8200cd580 2024-09-10T22:41:00.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.186+0000 7ff807fff640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff80c077f70 0x7ff80c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.189 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.186+0000 7ff807fff640 1 -- 172.21.15.66:0/3770282051 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7ff818099a60 con 0x7ff8200cd580 2024-09-10T22:41:00.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.186+0000 7ff81f7fe640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff80c077f70 0x7ff80c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.190 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.186+0000 7ff81f7fe640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff80c077f70 0x7ff80c07a430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff810033000 tx=0x7ff810004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:00.191 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.190+0000 7ff807fff640 1 -- 172.21.15.66:0/3770282051 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff818061f10 con 0x7ff8200cd580 2024-09-10T22:41:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.389+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff820144ec0 con 0x7ff8200cd580 2024-09-10T22:41:00.391 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.390+0000 7ff807fff640 1 -- 172.21.15.66:0/3770282051 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v53) ==== 80+0+122 (secure 0 0 0) 0x7ff818066940 con 0x7ff8200cd580 2024-09-10T22:41:00.391 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6835/3999038135 2024-09-11T22:38:48.643970+0000 2024-09-10T22:41:00.391 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6833/3999038135 2024-09-11T22:38:48.643970+0000 2024-09-10T22:41:00.392 INFO:teuthology.orchestra.run.smithi066.stderr:listed 2 entries 2024-09-10T22:41:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff80c077f70 msgr2=0x7ff80c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff80c077f70 0x7ff80c07a430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff810033000 tx=0x7ff810004730 comp rx=0 tx=0).stop 2024-09-10T22:41:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200cd580 msgr2=0x7ff820154540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200cd580 0x7ff820154540 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff81800c7a0 tx=0x7ff81800cc60 comp rx=0 tx=0).stop 2024-09-10T22:41:00.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 shutdown_connections 2024-09-10T22:41:00.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ff80c077f70 0x7ff80c07a430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff8200cd580 0x7ff820154540 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ff8200b5fc0 0x7ff82015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 --2- 172.21.15.66:0/3770282051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8200b5be0 0x7ff82015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 >> 172.21.15.66:0/3770282051 conn(0x7ff820069600 msgr2=0x7ff8200b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:00.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 shutdown_connections 2024-09-10T22:41:00.395 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.392+0000 7ff826233640 1 -- 172.21.15.66:0/3770282051 wait complete. 2024-09-10T22:41:00.404 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T22:41:00.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:00.404 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T22:41:00.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:41:00.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:00.435 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:41:00.491 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:41:00.491 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:41:00.491 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:00.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:41:00.511 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:41:00.512 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:41:00.512 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:00.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=930 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:41:00.548 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T22:41:00.548 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:41:00.548 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:00.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:41:00.570 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:41:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.693+0000 7f21f4add640 1 -- 172.21.15.66:0/2861426514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21dc003a00 msgr2=0x7f21dc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.693+0000 7f21f4add640 1 --2- 172.21.15.66:0/2861426514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21dc003a00 0x7f21dc003e20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c610 tx=0x7f21dc005830 comp rx=0 tx=0).stop 2024-09-10T22:41:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.694+0000 7f21eed9d640 1 -- 172.21.15.66:0/2861426514 reap_dead start 2024-09-10T22:41:00.696 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.694+0000 7f21f4add640 1 -- 172.21.15.66:0/2861426514 shutdown_connections 2024-09-10T22:41:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.694+0000 7f21f4add640 1 -- 172.21.15.66:0/2861426514 >> 172.21.15.66:0/2861426514 conn(0x7f21f00b1950 msgr2=0x7f21f00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.694+0000 7f21f4add640 1 -- 172.21.15.66:0/2861426514 shutdown_connections 2024-09-10T22:41:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.694+0000 7f21f4add640 1 -- 172.21.15.66:0/2861426514 wait complete. 2024-09-10T22:41:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.695+0000 7f21f4add640 1 Processor -- start 2024-09-10T22:41:00.697 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.695+0000 7f21f4add640 1 -- start start 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21f4add640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc003a00 0x7f21f014e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21f4add640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f009cf70 0x7f21f009d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21f4add640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21f00731d0 0x7f21f0073630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21f4add640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00ce1a0 con 0x7f21f00731d0 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21f4add640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00ce020 con 0x7f21dc003a00 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21f4add640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f21f00ce320 con 0x7f21f009cf70 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21f00731d0 0x7f21f0073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21f00731d0 0x7f21f0073630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39172/0 (socket says 172.21.15.66:39172) 2024-09-10T22:41:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 -- 172.21.15.66:0/131585233 learned_addr learned my addr 172.21.15.66:0/131585233 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21ee59c640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc003a00 0x7f21f014e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21edd9b640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f009cf70 0x7f21f009d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 -- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f009cf70 msgr2=0x7f21f009d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f009cf70 0x7f21f009d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 -- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc003a00 msgr2=0x7f21f014e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc003a00 0x7f21f014e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 -- 172.21.15.66:0/131585233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f01f60f0 con 0x7f21f00731d0 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.696+0000 7f21eed9d640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21f00731d0 0x7f21f0073630 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af850 tx=0x7f21e0011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.697+0000 7f21e6ffd640 1 -- 172.21.15.66:0/131585233 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000f920 con 0x7f21f00731d0 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.697+0000 7f21e6ffd640 1 -- 172.21.15.66:0/131585233 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e000fae0 con 0x7f21f00731d0 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.697+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f01f63d0 con 0x7f21f00731d0 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.697+0000 7f21e6ffd640 1 -- 172.21.15.66:0/131585233 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0005020 con 0x7f21f00731d0 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.697+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01f6990 con 0x7f21f00731d0 2024-09-10T22:41:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.698+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f0073b70 con 0x7f21f00731d0 2024-09-10T22:41:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.698+0000 7f21e6ffd640 1 -- 172.21.15.66:0/131585233 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f21e0021030 con 0x7f21f00731d0 2024-09-10T22:41:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.699+0000 7f21e6ffd640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f21d0077f70 0x7f21d007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.699+0000 7f21e6ffd640 1 -- 172.21.15.66:0/131585233 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f21e009a370 con 0x7f21f00731d0 2024-09-10T22:41:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.699+0000 7f21ee59c640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f21d0077f70 0x7f21d007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.700+0000 7f21ee59c640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f21d0077f70 0x7f21d007a430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21e8033000 tx=0x7f21e8001360 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.703+0000 7f21e6ffd640 1 -- 172.21.15.66:0/131585233 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f21e0062de0 con 0x7f21f00731d0 2024-09-10T22:41:00.924 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.923+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21f015be00 con 0x7f21f00731d0 2024-09-10T22:41:00.925 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.924+0000 7f21e6ffd640 1 -- 172.21.15.66:0/131585233 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f21e00626e0 con 0x7f21f00731d0 2024-09-10T22:41:00.925 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:00.925 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:41:00.925 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f21d0077f70 msgr2=0x7f21d007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f21d0077f70 0x7f21d007a430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f21e8033000 tx=0x7f21e8001360 comp rx=0 tx=0).stop 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21f00731d0 msgr2=0x7f21f0073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21f00731d0 0x7f21f0073630 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af850 tx=0x7f21e0011b10 comp rx=0 tx=0).stop 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 shutdown_connections 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f21d0077f70 0x7f21d007a430 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f21f00731d0 0x7f21f0073630 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f21f009cf70 0x7f21f009d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 --2- 172.21.15.66:0/131585233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21dc003a00 0x7f21f014e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 >> 172.21.15.66:0/131585233 conn(0x7f21f00b1950 msgr2=0x7f21f00c4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 shutdown_connections 2024-09-10T22:41:00.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:00.926+0000 7f21f4add640 1 -- 172.21.15.66:0/131585233 wait complete. 2024-09-10T22:41:00.938 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:41:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 -- 172.21.15.66:0/110973257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c56e0 msgr2=0x7fea5c1602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 --2- 172.21.15.66:0/110973257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c56e0 0x7fea5c1602f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea5c071ba0 tx=0x7fea4402a580 comp rx=0 tx=0).stop 2024-09-10T22:41:01.043 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 -- 172.21.15.66:0/110973257 shutdown_connections 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 --2- 172.21.15.66:0/110973257 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea5c0c6160 0x7fea5c0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 --2- 172.21.15.66:0/110973257 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c56e0 0x7fea5c1602f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 --2- 172.21.15.66:0/110973257 >> v2:172.21.15.94:3300/0 conn(0x7fea5c0b2de0 0x7fea5c073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 -- 172.21.15.66:0/110973257 >> 172.21.15.66:0/110973257 conn(0x7fea5c071380 msgr2=0x7fea5c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 -- 172.21.15.66:0/110973257 shutdown_connections 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.042+0000 7fea60a79640 1 -- 172.21.15.66:0/110973257 wait complete. 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.043+0000 7fea60a79640 1 Processor -- start 2024-09-10T22:41:01.044 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.043+0000 7fea60a79640 1 -- start start 2024-09-10T22:41:01.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea60a79640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea5c0b2de0 0x7fea5c1f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea60a79640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea5c0c56e0 0x7fea5c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea60a79640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c6160 0x7fea5c1f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea60a79640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c15c680 con 0x7fea5c0c6160 2024-09-10T22:41:01.046 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea60a79640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c15c500 con 0x7fea5c0c56e0 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea60a79640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c15c800 con 0x7fea5c0b2de0 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea5ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c6160 0x7fea5c1f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea5ad9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c6160 0x7fea5c1f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39196/0 (socket says 172.21.15.66:39196) 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea5ad9d640 1 -- 172.21.15.66:0/2360742911 learned_addr learned my addr 172.21.15.66:0/2360742911 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea59d9b640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea5c0c56e0 0x7fea5c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.044+0000 7fea5a59c640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea5c0b2de0 0x7fea5c1f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea5ad9d640 1 -- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea5c0b2de0 msgr2=0x7fea5c1f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea5ad9d640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea5c0b2de0 0x7fea5c1f5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea5ad9d640 1 -- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea5c0c56e0 msgr2=0x7fea5c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea5ad9d640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea5c0c56e0 0x7fea5c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea5ad9d640 1 -- 172.21.15.66:0/2360742911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c1f45a0 con 0x7fea5c0c6160 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea5ad9d640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c6160 0x7fea5c1f7b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d740 tx=0x7fea5400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea52ffd640 1 -- 172.21.15.66:0/2360742911 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54017070 con 0x7fea5c0c6160 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea52ffd640 1 -- 172.21.15.66:0/2360742911 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea54009d20 con 0x7fea5c0c6160 2024-09-10T22:41:01.047 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c1f85e0 con 0x7fea5c0c6160 2024-09-10T22:41:01.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea52ffd640 1 -- 172.21.15.66:0/2360742911 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54004280 con 0x7fea5c0c6160 2024-09-10T22:41:01.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.045+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1f4860 con 0x7fea5c0c6160 2024-09-10T22:41:01.048 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.046+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c0c6590 con 0x7fea5c0c6160 2024-09-10T22:41:01.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.047+0000 7fea52ffd640 1 -- 172.21.15.66:0/2360742911 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fea540035f0 con 0x7fea5c0c6160 2024-09-10T22:41:01.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.047+0000 7fea52ffd640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fea3c077f70 0x7fea3c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.047+0000 7fea52ffd640 1 -- 172.21.15.66:0/2360742911 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7fea54099dc0 con 0x7fea5c0c6160 2024-09-10T22:41:01.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.047+0000 7fea5a59c640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fea3c077f70 0x7fea3c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.049 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.048+0000 7fea5a59c640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fea3c077f70 0x7fea3c07a430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea4c005fd0 tx=0x7fea4c009560 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.052 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.051+0000 7fea52ffd640 1 -- 172.21.15.66:0/2360742911 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fea54062270 con 0x7fea5c0c6160 2024-09-10T22:41:01.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.273+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5c0b5860 con 0x7fea5c0c6160 2024-09-10T22:41:01.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.273+0000 7fea52ffd640 1 -- 172.21.15.66:0/2360742911 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7fea54066ca0 con 0x7fea5c0c6160 2024-09-10T22:41:01.275 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:01.275 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:41:01.275 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fea3c077f70 msgr2=0x7fea3c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fea3c077f70 0x7fea3c07a430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea4c005fd0 tx=0x7fea4c009560 comp rx=0 tx=0).stop 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c6160 msgr2=0x7fea5c1f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c6160 0x7fea5c1f7b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fea5400d740 tx=0x7fea5400dc00 comp rx=0 tx=0).stop 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 shutdown_connections 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fea3c077f70 0x7fea3c07a430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea5c0c6160 0x7fea5c1f7b90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea5c0c56e0 0x7fea5c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 --2- 172.21.15.66:0/2360742911 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fea5c0b2de0 0x7fea5c1f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.277 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.275+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 >> 172.21.15.66:0/2360742911 conn(0x7fea5c071380 msgr2=0x7fea5c070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:01.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.276+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 shutdown_connections 2024-09-10T22:41:01.278 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.276+0000 7fea60a79640 1 -- 172.21.15.66:0/2360742911 wait complete. 2024-09-10T22:41:01.287 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.391+0000 7febd8ebe640 1 -- 172.21.15.66:0/721002284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd40cdde0 msgr2=0x7febd415bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.391+0000 7febd8ebe640 1 --2- 172.21.15.66:0/721002284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd40cdde0 0x7febd415bc50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7febcc005fd0 tx=0x7febcc02aa90 comp rx=0 tx=0).stop 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.391+0000 7febd8ebe640 1 -- 172.21.15.66:0/721002284 shutdown_connections 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.391+0000 7febd8ebe640 1 --2- 172.21.15.66:0/721002284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd40cdde0 0x7febd415bc50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.391+0000 7febd8ebe640 1 --2- 172.21.15.66:0/721002284 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7febd415b0e0 0x7febd415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.391+0000 7febd8ebe640 1 -- 172.21.15.66:0/721002284 >> 172.21.15.66:0/721002284 conn(0x7febd40b2a60 msgr2=0x7febd40b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.392+0000 7febd8ebe640 1 -- 172.21.15.66:0/721002284 shutdown_connections 2024-09-10T22:41:01.393 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.392+0000 7febd8ebe640 1 -- 172.21.15.66:0/721002284 wait complete. 2024-09-10T22:41:01.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.393+0000 7febd8ebe640 1 Processor -- start 2024-09-10T22:41:01.394 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.393+0000 7febd8ebe640 1 -- start start 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd8ebe640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7febd40cdde0 0x7febd40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd8ebe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd415ad00 0x7febd4154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd8ebe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd415b0e0 0x7febd40ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd8ebe640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7febd4070290 con 0x7febd415ad00 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd8ebe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febd4070110 con 0x7febd415b0e0 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd8ebe640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7febd4070410 con 0x7febd40cdde0 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd415ad00 0x7febd4154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd415ad00 0x7febd4154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39212/0 (socket says 172.21.15.66:39212) 2024-09-10T22:41:01.396 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 -- 172.21.15.66:0/3803397594 learned_addr learned my addr 172.21.15.66:0/3803397594 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd259c640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7febd40cdde0 0x7febd40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd2d9d640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd415b0e0 0x7febd40ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 -- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7febd40cdde0 msgr2=0x7febd40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7febd40cdde0 0x7febd40b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 -- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd415b0e0 msgr2=0x7febd40ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd415b0e0 0x7febd40ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 -- 172.21.15.66:0/3803397594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd4158730 con 0x7febd415ad00 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd1d9b640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd415ad00 0x7febd4154270 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c970 tx=0x7febbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.397 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febcaffd640 1 -- 172.21.15.66:0/3803397594 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc007bb0 con 0x7febd415ad00 2024-09-10T22:41:01.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febcaffd640 1 -- 172.21.15.66:0/3803397594 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febbc005020 con 0x7febd415ad00 2024-09-10T22:41:01.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd4158a10 con 0x7febd415ad00 2024-09-10T22:41:01.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.394+0000 7febcaffd640 1 -- 172.21.15.66:0/3803397594 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc007e60 con 0x7febd415ad00 2024-09-10T22:41:01.398 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.395+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd41592b0 con 0x7febd415ad00 2024-09-10T22:41:01.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.396+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd40c7c20 con 0x7febd415ad00 2024-09-10T22:41:01.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.396+0000 7febcaffd640 1 -- 172.21.15.66:0/3803397594 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7febbc005470 con 0x7febd415ad00 2024-09-10T22:41:01.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.397+0000 7febcaffd640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7febb4077fa0 0x7febb407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.397+0000 7febcaffd640 1 -- 172.21.15.66:0/3803397594 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7febbc09a240 con 0x7febd415ad00 2024-09-10T22:41:01.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.397+0000 7febd259c640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7febb4077fa0 0x7febb407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.400 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.397+0000 7febd259c640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7febb4077fa0 0x7febb407a460 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febc4007fd0 tx=0x7febc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.401 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.401+0000 7febcaffd640 1 -- 172.21.15.66:0/3803397594 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7febd40c7c20 con 0x7febd415ad00 2024-09-10T22:41:01.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.586+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febd4158be0 con 0x7febd415ad00 2024-09-10T22:41:01.588 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.587+0000 7febcaffd640 1 -- 172.21.15.66:0/3803397594 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16929 (secure 0 0 0) 0x7febbc014030 con 0x7febd415ad00 2024-09-10T22:41:01.589 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:01.589 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":53,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:39:43.760611+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:38:37.601238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:38:38.924362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.12","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/3999038135":"2024-09-11T22:38:48.643970+0000","172.21.15.66:6833/3999038135":"2024-09-11T22:38:48.643970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7febb4077fa0 msgr2=0x7febb407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7febb4077fa0 0x7febb407a460 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febc4007fd0 tx=0x7febc4009040 comp rx=0 tx=0).stop 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd415ad00 msgr2=0x7febd4154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd415ad00 0x7febd4154270 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c970 tx=0x7febbc00ce30 comp rx=0 tx=0).stop 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 shutdown_connections 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7febb4077fa0 0x7febb407a460 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febd415b0e0 0x7febd40ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.591 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febd415ad00 0x7febd4154270 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 --2- 172.21.15.66:0/3803397594 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7febd40cdde0 0x7febd40b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 >> 172.21.15.66:0/3803397594 conn(0x7febd40b2a60 msgr2=0x7febd415e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:01.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 shutdown_connections 2024-09-10T22:41:01.592 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.589+0000 7febd8ebe640 1 -- 172.21.15.66:0/3803397594 wait complete. 2024-09-10T22:41:01.602 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:41:01.603 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:41:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 -- 172.21.15.66:0/1125647844 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 msgr2=0x7f2a2015ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 --2- 172.21.15.66:0/1125647844 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 0x7f2a2015ea70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005fd0 tx=0x7f2a1002b8b0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 -- 172.21.15.66:0/1125647844 shutdown_connections 2024-09-10T22:41:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 --2- 172.21.15.66:0/1125647844 >> v2:172.21.15.94:3300/0 conn(0x7f2a200c62f0 0x7f2a200cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 --2- 172.21.15.66:0/1125647844 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 0x7f2a2015ea70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 -- 172.21.15.66:0/1125647844 >> 172.21.15.66:0/1125647844 conn(0x7f2a200b1950 msgr2=0x7f2a200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:01.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 -- 172.21.15.66:0/1125647844 shutdown_connections 2024-09-10T22:41:01.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.708+0000 7f2a25662640 1 -- 172.21.15.66:0/1125647844 wait complete. 2024-09-10T22:41:01.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.709+0000 7f2a25662640 1 Processor -- start 2024-09-10T22:41:01.710 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.709+0000 7f2a25662640 1 -- start start 2024-09-10T22:41:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a25662640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a200c62f0 0x7f2a201f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a25662640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a200ce0f0 0x7f2a201f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a25662640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 0x7f2a200c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a25662640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200cc7c0 con 0x7f2a200ce0f0 2024-09-10T22:41:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a25662640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200cc940 con 0x7f2a200c62f0 2024-09-10T22:41:01.712 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a25662640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f2a200ccac0 con 0x7f2a2015db50 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a200ce0f0 0x7f2a201f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a200ce0f0 0x7f2a201f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39224/0 (socket says 172.21.15.66:39224) 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 -- 172.21.15.66:0/714153976 learned_addr learned my addr 172.21.15.66:0/714153976 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1f7fe640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 0x7f2a200c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1effd640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a200c62f0 0x7f2a201f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 -- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 msgr2=0x7f2a200c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 0x7f2a200c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 -- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a200c62f0 msgr2=0x7f2a201f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a200c62f0 0x7f2a201f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.710+0000 7f2a1e7fc640 1 -- 172.21.15.66:0/714153976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a201f6d40 con 0x7f2a200ce0f0 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.711+0000 7f2a1e7fc640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a200ce0f0 0x7f2a201f3740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800e9c0 tx=0x7f2a0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.711+0000 7f2a177fe640 1 -- 172.21.15.66:0/714153976 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800cd80 con 0x7f2a200ce0f0 2024-09-10T22:41:01.713 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.711+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a201f7020 con 0x7f2a200ce0f0 2024-09-10T22:41:01.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.711+0000 7f2a177fe640 1 -- 172.21.15.66:0/714153976 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a08004020 con 0x7f2a200ce0f0 2024-09-10T22:41:01.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.711+0000 7f2a177fe640 1 -- 172.21.15.66:0/714153976 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08004450 con 0x7f2a200ce0f0 2024-09-10T22:41:01.714 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.711+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a201f3240 con 0x7f2a200ce0f0 2024-09-10T22:41:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.712+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a200cdbb0 con 0x7f2a200ce0f0 2024-09-10T22:41:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.713+0000 7f2a177fe640 1 -- 172.21.15.66:0/714153976 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2a080035f0 con 0x7f2a200ce0f0 2024-09-10T22:41:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.713+0000 7f2a177fe640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2a00077f70 0x7f2a0007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:01.715 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.713+0000 7f2a177fe640 1 -- 172.21.15.66:0/714153976 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f2a0809a830 con 0x7f2a200ce0f0 2024-09-10T22:41:01.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.713+0000 7f2a1effd640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2a00077f70 0x7f2a0007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:01.716 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.714+0000 7f2a1effd640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2a00077f70 0x7f2a0007a430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a10035000 tx=0x7f2a10004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:01.718 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.717+0000 7f2a177fe640 1 -- 172.21.15.66:0/714153976 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2a08062bd0 con 0x7f2a200ce0f0 2024-09-10T22:41:01.942 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.941+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a201f71f0 con 0x7f2a200ce0f0 2024-09-10T22:41:01.943 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.941+0000 7f2a177fe640 1 -- 172.21.15.66:0/714153976 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f2a08067600 con 0x7f2a200ce0f0 2024-09-10T22:41:01.943 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:01.943 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:41:01.943 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:41:01.944 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2a00077f70 msgr2=0x7f2a0007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2a00077f70 0x7f2a0007a430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a10035000 tx=0x7f2a10004730 comp rx=0 tx=0).stop 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a200ce0f0 msgr2=0x7f2a201f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a200ce0f0 0x7f2a201f3740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800e9c0 tx=0x7f2a0800ee80 comp rx=0 tx=0).stop 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 shutdown_connections 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f2a00077f70 0x7f2a0007a430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f2a2015db50 0x7f2a200c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2a200ce0f0 0x7f2a201f3740 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 --2- 172.21.15.66:0/714153976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a200c62f0 0x7f2a201f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 >> 172.21.15.66:0/714153976 conn(0x7f2a200b1950 msgr2=0x7f2a2006f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 shutdown_connections 2024-09-10T22:41:01.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:01.943+0000 7f2a25662640 1 -- 172.21.15.66:0/714153976 wait complete. 2024-09-10T22:41:01.955 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:41:02.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 -- 172.21.15.66:0/4062083796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406fcb0 msgr2=0x7f65a4070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 --2- 172.21.15.66:0/4062083796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406fcb0 0x7f65a4070140 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f659c005fd0 tx=0x7f659c02aa10 comp rx=0 tx=0).stop 2024-09-10T22:41:02.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 -- 172.21.15.66:0/4062083796 shutdown_connections 2024-09-10T22:41:02.060 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 --2- 172.21.15.66:0/4062083796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406fcb0 0x7f65a4070140 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 --2- 172.21.15.66:0/4062083796 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a406cb70 0x7f65a406cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 -- 172.21.15.66:0/4062083796 >> 172.21.15.66:0/4062083796 conn(0x7f65a40b2a60 msgr2=0x7f65a40b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:02.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 -- 172.21.15.66:0/4062083796 shutdown_connections 2024-09-10T22:41:02.061 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.059+0000 7f65aaebd640 1 -- 172.21.15.66:0/4062083796 wait complete. 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.060+0000 7f65aaebd640 1 Processor -- start 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.060+0000 7f65aaebd640 1 -- start start 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65aaebd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406cb70 0x7f65a40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65aaebd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a406fcb0 0x7f65a40b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65aaebd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a40d0780 0x7f65a40c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65aaebd640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4070800 con 0x7f65a406cb70 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65aaebd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65a4070680 con 0x7f65a40d0780 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65aaebd640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f65a4070980 con 0x7f65a406fcb0 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406cb70 0x7f65a40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.063 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406cb70 0x7f65a40b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39242/0 (socket says 172.21.15.66:39242) 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 -- 172.21.15.66:0/2465622790 learned_addr learned my addr 172.21.15.66:0/2465622790 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f659bfff640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a406fcb0 0x7f65a40b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a945a640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a40d0780 0x7f65a40c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 -- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a406fcb0 msgr2=0x7f65a40b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a406fcb0 0x7f65a40b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 -- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a40d0780 msgr2=0x7f65a40c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a40d0780 0x7f65a40c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 -- 172.21.15.66:0/2465622790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a40cb7e0 con 0x7f65a406cb70 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.061+0000 7f65a8c59640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406cb70 0x7f65a40b5520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c3a0 tx=0x7f65a000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.062+0000 7f65997fa640 1 -- 172.21.15.66:0/2465622790 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0017070 con 0x7f65a406cb70 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.062+0000 7f65997fa640 1 -- 172.21.15.66:0/2465622790 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65a00053c0 con 0x7f65a406cb70 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.062+0000 7f65997fa640 1 -- 172.21.15.66:0/2465622790 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0003850 con 0x7f65a406cb70 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.062+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a40cba50 con 0x7f65a406cb70 2024-09-10T22:41:02.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.062+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a40cc010 con 0x7f65a406cb70 2024-09-10T22:41:02.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.063+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a4070e30 con 0x7f65a406cb70 2024-09-10T22:41:02.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.063+0000 7f65997fa640 1 -- 172.21.15.66:0/2465622790 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f65a00039f0 con 0x7f65a406cb70 2024-09-10T22:41:02.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.064+0000 7f65997fa640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6590077f70 0x7f659007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.064+0000 7f65997fa640 1 -- 172.21.15.66:0/2465622790 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7f65a0099380 con 0x7f65a406cb70 2024-09-10T22:41:02.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.064+0000 7f659bfff640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6590077f70 0x7f659007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.066 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.065+0000 7f659bfff640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6590077f70 0x7f659007a430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f65a40b6290 tx=0x7f659400b390 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:02.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.068+0000 7f65997fa640 1 -- 172.21.15.66:0/2465622790 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f65a00617a0 con 0x7f65a406cb70 2024-09-10T22:41:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.295+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65a40b25b0 con 0x7f65a406cb70 2024-09-10T22:41:02.296 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.295+0000 7f65997fa640 1 -- 172.21.15.66:0/2465622790 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5215 (secure 0 0 0) 0x7f65a00661d0 con 0x7f65a406cb70 2024-09-10T22:41:02.296 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:02.297 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":39,"btime":"2024-09-10T22:38:52:670641+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:38:39.959431+0000","modified":"2024-09-10T22:38:52.670633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5454":{"gid":5454,"name":"c","rank":0,"incarnation":35,"state":"up:active","state_seq":5,"addr":"172.21.15.66:6834/917223506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":917223506},{"type":"v1","addr":"172.21.15.66:6834","nonce":917223506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5454,"qdb_cluster":[5454]},"id":3}]} 2024-09-10T22:41:02.297 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 39 2024-09-10T22:41:02.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.297+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6590077f70 msgr2=0x7f659007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.299 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.297+0000 7f65aaebd640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6590077f70 0x7f659007a430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f65a40b6290 tx=0x7f659400b390 comp rx=0 tx=0).stop 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406cb70 msgr2=0x7f65a40b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406cb70 0x7f65a40b5520 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c3a0 tx=0x7f65a000c860 comp rx=0 tx=0).stop 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 shutdown_connections 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6590077f70 0x7f659007a430 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65a40d0780 0x7f65a40c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f65a406fcb0 0x7f65a40b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 --2- 172.21.15.66:0/2465622790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65a406cb70 0x7f65a40b5520 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 >> 172.21.15.66:0/2465622790 conn(0x7f65a40b2a60 msgr2=0x7f65a40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 shutdown_connections 2024-09-10T22:41:02.300 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.298+0000 7f65aaebd640 1 -- 172.21.15.66:0/2465622790 wait complete. 2024-09-10T22:41:02.310 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:41:02.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.413+0000 7fc1091ad640 1 -- 172.21.15.66:0/2212461955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c6160 msgr2=0x7fc1040c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.413+0000 7fc1091ad640 1 --2- 172.21.15.66:0/2212461955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c6160 0x7fc1040c65b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005fd0 tx=0x7fc0f402a520 comp rx=0 tx=0).stop 2024-09-10T22:41:02.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.413+0000 7fc1091ad640 1 -- 172.21.15.66:0/2212461955 shutdown_connections 2024-09-10T22:41:02.415 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.413+0000 7fc1091ad640 1 --2- 172.21.15.66:0/2212461955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c6160 0x7fc1040c65b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.414+0000 7fc1091ad640 1 --2- 172.21.15.66:0/2212461955 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1040c56e0 0x7fc104073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.414+0000 7fc1091ad640 1 -- 172.21.15.66:0/2212461955 >> 172.21.15.66:0/2212461955 conn(0x7fc104071380 msgr2=0x7fc10406cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:02.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.414+0000 7fc1091ad640 1 -- 172.21.15.66:0/2212461955 shutdown_connections 2024-09-10T22:41:02.416 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.414+0000 7fc1091ad640 1 -- 172.21.15.66:0/2212461955 wait complete. 2024-09-10T22:41:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.415+0000 7fc1091ad640 1 Processor -- start 2024-09-10T22:41:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.415+0000 7fc1091ad640 1 -- start start 2024-09-10T22:41:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc1091ad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1040b2de0 0x7fc1040cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc1091ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c56e0 0x7fc104152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.418 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc1091ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1040c6160 0x7fc104152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc1091ad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc10415c890 con 0x7fc1040c56e0 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc1091ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc10415c710 con 0x7fc1040c6160 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc1091ad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc10415ca10 con 0x7fc1040b2de0 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c56e0 0x7fc104152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c56e0 0x7fc104152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39262/0 (socket says 172.21.15.66:39262) 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 -- 172.21.15.66:0/1757492398 learned_addr learned my addr 172.21.15.66:0/1757492398 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc102d9d640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1040b2de0 0x7fc1040cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10359e640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1040c6160 0x7fc104152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 -- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1040b2de0 msgr2=0x7fc1040cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1040b2de0 0x7fc1040cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 -- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1040c6160 msgr2=0x7fc104152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1040c6160 0x7fc104152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 -- 172.21.15.66:0/1757492398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1041f44b0 con 0x7fc1040c56e0 2024-09-10T22:41:02.419 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.416+0000 7fc10259c640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c56e0 0x7fc104152150 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c920 tx=0x7fc0ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.417+0000 7fc0fb7fe640 1 -- 172.21.15.66:0/1757492398 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec007c70 con 0x7fc1040c56e0 2024-09-10T22:41:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.417+0000 7fc0fb7fe640 1 -- 172.21.15.66:0/1757492398 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0ec00f450 con 0x7fc1040c56e0 2024-09-10T22:41:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.417+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1041f4730 con 0x7fc1040c56e0 2024-09-10T22:41:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.417+0000 7fc0fb7fe640 1 -- 172.21.15.66:0/1757492398 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00f8a0 con 0x7fc1040c56e0 2024-09-10T22:41:02.420 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.417+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1041f4cf0 con 0x7fc1040c56e0 2024-09-10T22:41:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.418+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10415ce90 con 0x7fc1040c56e0 2024-09-10T22:41:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.418+0000 7fc0fb7fe640 1 -- 172.21.15.66:0/1757492398 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc0ec020030 con 0x7fc1040c56e0 2024-09-10T22:41:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.419+0000 7fc0fb7fe640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc0e4077f70 0x7fc0e407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.419+0000 7fc0fb7fe640 1 -- 172.21.15.66:0/1757492398 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7fc0ec09a1e0 con 0x7fc1040c56e0 2024-09-10T22:41:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.419+0000 7fc102d9d640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc0e4077f70 0x7fc0e407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.421 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.420+0000 7fc102d9d640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc0e4077f70 0x7fc0e407a430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1041530a0 tx=0x7fc0fc00d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:02.424 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.423+0000 7fc0fb7fe640 1 -- 172.21.15.66:0/1757492398 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc1041f44b0 con 0x7fc1040c56e0 2024-09-10T22:41:02.608 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.607+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc10406e1b0 con 0x7fc1040c56e0 2024-09-10T22:41:02.609 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.608+0000 7fc0fb7fe640 1 -- 172.21.15.66:0/1757492398 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16929 (secure 0 0 0) 0x7fc0ec014030 con 0x7fc1040c56e0 2024-09-10T22:41:02.609 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:02.610 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":53,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:39:43.760611+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:38:37.601238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T22:38:38.924362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.12","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6835/3999038135":"2024-09-11T22:38:48.643970+0000","172.21.15.66:6833/3999038135":"2024-09-11T22:38:48.643970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:41:02.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc0e4077f70 msgr2=0x7fc0e407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc0e4077f70 0x7fc0e407a430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1041530a0 tx=0x7fc0fc00d040 comp rx=0 tx=0).stop 2024-09-10T22:41:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c56e0 msgr2=0x7fc104152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c56e0 0x7fc104152150 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c920 tx=0x7fc0ec00cde0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 shutdown_connections 2024-09-10T22:41:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc0e4077f70 0x7fc0e407a430 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1040c6160 0x7fc104152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1040c56e0 0x7fc104152150 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 --2- 172.21.15.66:0/1757492398 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc1040b2de0 0x7fc1040cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 >> 172.21.15.66:0/1757492398 conn(0x7fc104071380 msgr2=0x7fc1040cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 shutdown_connections 2024-09-10T22:41:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.610+0000 7fc1091ad640 1 -- 172.21.15.66:0/1757492398 wait complete. 2024-09-10T22:41:02.622 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 -- 172.21.15.66:0/2085861984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0d0840 msgr2=0x7fe73c0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2085861984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0d0840 0x7fe73c0d0cd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe73c08e6b0 tx=0x7fe73402a980 comp rx=0 tx=0).stop 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 -- 172.21.15.66:0/2085861984 shutdown_connections 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2085861984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0d0840 0x7fe73c0d0cd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2085861984 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe73c0cf150 0x7fe73c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2085861984 >> v2:172.21.15.94:3300/0 conn(0x7fe73c15e310 0x7fe73c0cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 -- 172.21.15.66:0/2085861984 >> 172.21.15.66:0/2085861984 conn(0x7fe73c0d4100 msgr2=0x7fe73c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 -- 172.21.15.66:0/2085861984 shutdown_connections 2024-09-10T22:41:02.730 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.727+0000 7fe7429a2640 1 -- 172.21.15.66:0/2085861984 wait complete. 2024-09-10T22:41:02.731 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.729+0000 7fe7429a2640 1 Processor -- start 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.729+0000 7fe7429a2640 1 -- start start 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe7429a2640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0cf150 0x7fe73c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe7429a2640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe73c0d0840 0x7fe73c0b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe7429a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe73c15e310 0x7fe73c0b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe7429a2640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c0d1390 con 0x7fe73c0cf150 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe7429a2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c0d1210 con 0x7fe73c15e310 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe7429a2640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe73c0d1510 con 0x7fe73c0d0840 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0cf150 0x7fe73c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0cf150 0x7fe73c0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39280/0 (socket says 172.21.15.66:39280) 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 -- 172.21.15.66:0/2947491641 learned_addr learned my addr 172.21.15.66:0/2947491641 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73b7fe640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe73c0d0840 0x7fe73c0b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe740f3f640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe73c15e310 0x7fe73c0b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.732 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 -- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe73c0d0840 msgr2=0x7fe73c0b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe73c0d0840 0x7fe73c0b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 -- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe73c15e310 msgr2=0x7fe73c0b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe73c15e310 0x7fe73c0b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 -- 172.21.15.66:0/2947491641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c0cb1d0 con 0x7fe73c0cf150 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe73bfff640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0cf150 0x7fe73c0cc270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00d770 tx=0x7fe72c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe738ff9640 1 -- 172.21.15.66:0/2947491641 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c017070 con 0x7fe73c0cf150 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe738ff9640 1 -- 172.21.15.66:0/2947491641 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe72c004310 con 0x7fe73c0cf150 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe738ff9640 1 -- 172.21.15.66:0/2947491641 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c009c30 con 0x7fe73c0cf150 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.730+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c0cb4b0 con 0x7fe73c0cf150 2024-09-10T22:41:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.731+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c0cbd50 con 0x7fe73c0cf150 2024-09-10T22:41:02.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.731+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c0c7880 con 0x7fe73c0cf150 2024-09-10T22:41:02.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.732+0000 7fe738ff9640 1 -- 172.21.15.66:0/2947491641 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe72c005020 con 0x7fe73c0cf150 2024-09-10T22:41:02.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.733+0000 7fe738ff9640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe728077f70 0x7fe72807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:02.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.733+0000 7fe738ff9640 1 -- 172.21.15.66:0/2947491641 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5853+0+0 (secure 0 0 0) 0x7fe72c099fb0 con 0x7fe73c0cf150 2024-09-10T22:41:02.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.733+0000 7fe73b7fe640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe728077f70 0x7fe72807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:02.734 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.733+0000 7fe73b7fe640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe728077f70 0x7fe72807a430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0a2230 tx=0x7fe730005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:02.737 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.737+0000 7fe738ff9640 1 -- 172.21.15.66:0/2947491641 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe73c0c7880 con 0x7fe73c0cf150 2024-09-10T22:41:02.959 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.958+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe73c0d19d0 con 0x7fe73c0cf150 2024-09-10T22:41:02.960 INFO:tasks.ceph.mon.a.smithi066.stderr:2024-09-10T22:41:02.959+0000 7fe21e4b6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:41:02.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.960+0000 7fe738ff9640 1 -- 172.21.15.66:0/2947491641 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7fe72c020020 con 0x7fe73c0cf150 2024-09-10T22:41:02.961 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe728077f70 msgr2=0x7fe72807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe728077f70 0x7fe72807a430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0a2230 tx=0x7fe730005f90 comp rx=0 tx=0).stop 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0cf150 msgr2=0x7fe73c0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0cf150 0x7fe73c0cc270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00d770 tx=0x7fe72c00dc30 comp rx=0 tx=0).stop 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 shutdown_connections 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe728077f70 0x7fe72807a430 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe73c15e310 0x7fe73c0b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe73c0d0840 0x7fe73c0b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 --2- 172.21.15.66:0/2947491641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe73c0cf150 0x7fe73c0cc270 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 >> 172.21.15.66:0/2947491641 conn(0x7fe73c0d4100 msgr2=0x7fe73c0b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 shutdown_connections 2024-09-10T22:41:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:02.963+0000 7fe7429a2640 1 -- 172.21.15.66:0/2947491641 wait complete. 2024-09-10T22:41:02.969 INFO:tasks.ceph.mds.c.smithi066.stderr: -6358> 2024-09-10T22:38:53.653+0000 7f371001e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:41:02.979 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:41:02.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:02.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:03.003 INFO:tasks.ceph.mds.c.smithi066.stdout:starting mds.c at 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 -- 172.21.15.66:0/1820966169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c161ab0 msgr2=0x7fc71c0d4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 --2- 172.21.15.66:0/1820966169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c161ab0 0x7fc71c0d4ca0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009830 tx=0x7fc70c02a480 comp rx=0 tx=0).stop 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 -- 172.21.15.66:0/1820966169 shutdown_connections 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 --2- 172.21.15.66:0/1820966169 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc71c06de00 0x7fc71c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 --2- 172.21.15.66:0/1820966169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c161ab0 0x7fc71c0d4ca0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 -- 172.21.15.66:0/1820966169 >> 172.21.15.66:0/1820966169 conn(0x7fc71c0d41b0 msgr2=0x7fc71c0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 -- 172.21.15.66:0/1820966169 shutdown_connections 2024-09-10T22:41:03.091 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.089+0000 7fc722008640 1 -- 172.21.15.66:0/1820966169 wait complete. 2024-09-10T22:41:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.091+0000 7fc722008640 1 Processor -- start 2024-09-10T22:41:03.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.091+0000 7fc722008640 1 -- start start 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc722008640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06de00 0x7fc71c0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc722008640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc71c0d20b0 0x7fc71c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc722008640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc71c161ab0 0x7fc71c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc722008640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c0cf7d0 con 0x7fc71c06de00 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc722008640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c0d2710 con 0x7fc71c161ab0 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc722008640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c0cf950 con 0x7fc71c0d20b0 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06de00 0x7fc71c0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71affd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc71c0d20b0 0x7fc71c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:03.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06de00 0x7fc71c0ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39320/0 (socket says 172.21.15.66:39320) 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 -- 172.21.15.66:0/2266612554 learned_addr learned my addr 172.21.15.66:0/2266612554 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71bfff640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc71c161ab0 0x7fc71c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 -- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc71c0d20b0 msgr2=0x7fc71c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc71c0d20b0 0x7fc71c0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 -- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc71c161ab0 msgr2=0x7fc71c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc71c161ab0 0x7fc71c0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 -- 172.21.15.66:0/2266612554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c0cc3a0 con 0x7fc71c06de00 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc71b7fe640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06de00 0x7fc71c0ca760 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009950 tx=0x7fc70c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.092+0000 7fc703fff640 1 -- 172.21.15.66:0/2266612554 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c031d60 con 0x7fc71c06de00 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.093+0000 7fc703fff640 1 -- 172.21.15.66:0/2266612554 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc70c033e10 con 0x7fc71c06de00 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.093+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c0b5eb0 con 0x7fc71c06de00 2024-09-10T22:41:03.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.093+0000 7fc703fff640 1 -- 172.21.15.66:0/2266612554 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c005260 con 0x7fc71c06de00 2024-09-10T22:41:03.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.093+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c0caca0 con 0x7fc71c06de00 2024-09-10T22:41:03.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.094+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c0d4ca0 con 0x7fc71c06de00 2024-09-10T22:41:03.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.094+0000 7fc703fff640 1 -- 172.21.15.66:0/2266612554 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc70c005400 con 0x7fc71c06de00 2024-09-10T22:41:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.095+0000 7fc703fff640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc708077f70 0x7fc70807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.095+0000 7fc703fff640 1 -- 172.21.15.66:0/2266612554 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5939+0+0 (secure 0 0 0) 0x7fc70c0bae70 con 0x7fc71c06de00 2024-09-10T22:41:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.095+0000 7fc71affd640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc708077f70 0x7fc70807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:03.097 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.096+0000 7fc71affd640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc708077f70 0x7fc70807a430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc710007fd0 tx=0x7fc710009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:03.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.099+0000 7fc703fff640 1 -- 172.21.15.66:0/2266612554 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc71c0b5eb0 con 0x7fc71c06de00 2024-09-10T22:41:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.329+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc71c0d2be0 con 0x7fc71c06de00 2024-09-10T22:41:03.962 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.961+0000 7fc703fff640 1 -- 172.21.15.66:0/2266612554 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7fc70c03f030 con 0x7fc71c06de00 2024-09-10T22:41:03.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc708077f70 msgr2=0x7fc70807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:03.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc708077f70 0x7fc70807a430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc710007fd0 tx=0x7fc710009040 comp rx=0 tx=0).stop 2024-09-10T22:41:03.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06de00 msgr2=0x7fc71c0ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:03.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06de00 0x7fc71c0ca760 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009950 tx=0x7fc70c033c20 comp rx=0 tx=0).stop 2024-09-10T22:41:03.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 shutdown_connections 2024-09-10T22:41:03.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc708077f70 0x7fc70807a430 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.965 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc71c161ab0 0x7fc71c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc71c0d20b0 0x7fc71c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 --2- 172.21.15.66:0/2266612554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc71c06de00 0x7fc71c0ca760 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:03.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 >> 172.21.15.66:0/2266612554 conn(0x7fc71c0d41b0 msgr2=0x7fc71c0b8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:03.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 shutdown_connections 2024-09-10T22:41:03.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:03.964+0000 7fc722008640 1 -- 172.21.15.66:0/2266612554 wait complete. 2024-09-10T22:41:03.977 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:41:04.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.088+0000 7f0dab733640 1 -- 172.21.15.66:0/690163039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4161890 msgr2=0x7f0da409b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:04.089 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.088+0000 7f0dab733640 1 --2- 172.21.15.66:0/690163039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4161890 0x7f0da409b590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0da0005fd0 tx=0x7f0da002a5d0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.088+0000 7f0dab733640 1 -- 172.21.15.66:0/690163039 shutdown_connections 2024-09-10T22:41:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.088+0000 7f0dab733640 1 --2- 172.21.15.66:0/690163039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4161890 0x7f0da409b590 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.088+0000 7f0dab733640 1 -- 172.21.15.66:0/690163039 >> 172.21.15.66:0/690163039 conn(0x7f0da409aa70 msgr2=0x7f0da409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.088+0000 7f0dab733640 1 -- 172.21.15.66:0/690163039 shutdown_connections 2024-09-10T22:41:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.088+0000 7f0dab733640 1 -- 172.21.15.66:0/690163039 wait complete. 2024-09-10T22:41:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.089+0000 7f0dab733640 1 Processor -- start 2024-09-10T22:41:04.090 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.089+0000 7f0dab733640 1 -- start start 2024-09-10T22:41:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0dab733640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0da406e130 0x7f0da40cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0dab733640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40b2250 0x7f0da40cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0dab733640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0da4161890 0x7f0da41f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0dab733640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40b2f50 con 0x7f0da40b2250 2024-09-10T22:41:04.092 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0dab733640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40b2dd0 con 0x7f0da4161890 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0dab733640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f0da40b30d0 con 0x7f0da406e130 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0da8cce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40b2250 0x7f0da40cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0da8cce640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40b2250 0x7f0da40cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36714/0 (socket says 172.21.15.66:36714) 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0da8cce640 1 -- 172.21.15.66:0/4116868648 learned_addr learned my addr 172.21.15.66:0/4116868648 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0da94cf640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0da406e130 0x7f0da40cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0da8cce640 1 -- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0da406e130 msgr2=0x7f0da40cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0da8cce640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0da406e130 0x7f0da40cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0da8cce640 1 -- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0da4161890 msgr2=0x7f0da41f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.090+0000 7f0da9cd0640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0da4161890 0x7f0da41f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0da8cce640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0da4161890 0x7f0da41f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0da8cce640 1 -- 172.21.15.66:0/4116868648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da40c9840 con 0x7f0da40b2250 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0da94cf640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0da406e130 0x7f0da40cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:04.093 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0da9cd0640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0da4161890 0x7f0da41f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0da8cce640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40b2250 0x7f0da40cb750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c410 tx=0x7f0d9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0d99ffb640 1 -- 172.21.15.66:0/4116868648 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94017070 con 0x7f0da40b2250 2024-09-10T22:41:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0d99ffb640 1 -- 172.21.15.66:0/4116868648 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d940035f0 con 0x7f0da40b2250 2024-09-10T22:41:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0d99ffb640 1 -- 172.21.15.66:0/4116868648 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94003a80 con 0x7f0da40b2250 2024-09-10T22:41:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da40c9b20 con 0x7f0da40b2250 2024-09-10T22:41:04.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.091+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da40ca0e0 con 0x7f0da40b2250 2024-09-10T22:41:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.092+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da409b590 con 0x7f0da40b2250 2024-09-10T22:41:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.093+0000 7f0d99ffb640 1 -- 172.21.15.66:0/4116868648 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0d94003c20 con 0x7f0da40b2250 2024-09-10T22:41:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.093+0000 7f0d99ffb640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d88077f70 0x7f0d8807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.093+0000 7f0d99ffb640 1 -- 172.21.15.66:0/4116868648 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5939+0+0 (secure 0 0 0) 0x7f0d94099460 con 0x7f0da40b2250 2024-09-10T22:41:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.094+0000 7f0da94cf640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d88077f70 0x7f0d8807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:04.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.094+0000 7f0da94cf640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d88077f70 0x7f0d8807a430 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0da0033000 tx=0x7f0da0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:04.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.097+0000 7f0d99ffb640 1 -- 172.21.15.66:0/4116868648 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0d94061f00 con 0x7f0da40b2250 2024-09-10T22:41:04.285 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.284+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0da40b7ed0 con 0x7f0da40b2250 2024-09-10T22:41:04.966 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.965+0000 7f0d99ffb640 1 -- 172.21.15.66:0/4116868648 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) ==== 181+0+0 (secure 0 0 0) 0x7f0d94061800 con 0x7f0da40b2250 2024-09-10T22:41:04.967 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:41:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d88077f70 msgr2=0x7f0d8807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d88077f70 0x7f0d8807a430 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0da0033000 tx=0x7f0da0004730 comp rx=0 tx=0).stop 2024-09-10T22:41:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40b2250 msgr2=0x7f0da40cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40b2250 0x7f0da40cb750 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c410 tx=0x7f0d9400c8d0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.970 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 shutdown_connections 2024-09-10T22:41:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f0d88077f70 0x7f0d8807a430 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0da4161890 0x7f0da41f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40b2250 0x7f0da40cb750 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c410 tx=0x7f0d9400c8d0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 --2- 172.21.15.66:0/4116868648 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f0da406e130 0x7f0da40cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 >> 172.21.15.66:0/4116868648 conn(0x7f0da409aa70 msgr2=0x7f0da40b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 shutdown_connections 2024-09-10T22:41:04.971 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:04.969+0000 7f0dab733640 1 -- 172.21.15.66:0/4116868648 wait complete. 2024-09-10T22:41:04.983 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:41:05.094 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.093+0000 7f05f7240640 1 -- 172.21.15.66:0/2293708022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05dc003a00 msgr2=0x7f05dc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:05.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.093+0000 7f05f7240640 1 --2- 172.21.15.66:0/2293708022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05dc003a00 0x7f05dc003e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f05e000c590 tx=0x7f05dc005830 comp rx=0 tx=0).stop 2024-09-10T22:41:05.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.093+0000 7f05f57dd640 1 -- 172.21.15.66:0/2293708022 reap_dead start 2024-09-10T22:41:05.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.093+0000 7f05f7240640 1 -- 172.21.15.66:0/2293708022 shutdown_connections 2024-09-10T22:41:05.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.093+0000 7f05f7240640 1 -- 172.21.15.66:0/2293708022 >> 172.21.15.66:0/2293708022 conn(0x7f05f008dac0 msgr2=0x7f05f009d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:05.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.093+0000 7f05f7240640 1 -- 172.21.15.66:0/2293708022 shutdown_connections 2024-09-10T22:41:05.095 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.093+0000 7f05f7240640 1 -- 172.21.15.66:0/2293708022 wait complete. 2024-09-10T22:41:05.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.094+0000 7f05f7240640 1 Processor -- start 2024-09-10T22:41:05.096 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.094+0000 7f05f7240640 1 -- start start 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.095+0000 7f05f7240640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05dc003a00 0x7f05f015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.095+0000 7f05f7240640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f00b3630 0x7f05f00b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.095+0000 7f05f7240640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05f015c450 0x7f05f015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.095+0000 7f05f7240640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f05f00d4350 con 0x7f05f015c450 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.095+0000 7f05f7240640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f05f00d41d0 con 0x7f05f00b3630 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.095+0000 7f05f7240640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f05f00d44d0 con 0x7f05dc003a00 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.095+0000 7f05f57dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05f015c450 0x7f05f015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05f015c450 0x7f05f015c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36742/0 (socket says 172.21.15.66:36742) 2024-09-10T22:41:05.098 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 -- 172.21.15.66:0/872238831 learned_addr learned my addr 172.21.15.66:0/872238831 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f4fdc640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05dc003a00 0x7f05f015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05effff640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f00b3630 0x7f05f00b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 -- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05dc003a00 msgr2=0x7f05f015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05dc003a00 0x7f05f015fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 -- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f00b3630 msgr2=0x7f05f00b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f00b3630 0x7f05f00b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 -- 172.21.15.66:0/872238831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f01f6560 con 0x7f05f015c450 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f57dd640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05f015c450 0x7f05f015c880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f05f00d1440 tx=0x7f05e8011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05ed7fa640 1 -- 172.21.15.66:0/872238831 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e800fca0 con 0x7f05f015c450 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05ed7fa640 1 -- 172.21.15.66:0/872238831 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05e8011e80 con 0x7f05f015c450 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f01f6840 con 0x7f05f015c450 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05ed7fa640 1 -- 172.21.15.66:0/872238831 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8004250 con 0x7f05f015c450 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.096+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01f6e00 con 0x7f05f015c450 2024-09-10T22:41:05.099 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.097+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f006d440 con 0x7f05f015c450 2024-09-10T22:41:05.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.098+0000 7f05ed7fa640 1 -- 172.21.15.66:0/872238831 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f05e80035f0 con 0x7f05f015c450 2024-09-10T22:41:05.100 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.098+0000 7f05ed7fa640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f05d0077f70 0x7f05d007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:05.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.098+0000 7f05ed7fa640 1 -- 172.21.15.66:0/872238831 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5501+0+0 (secure 0 0 0) 0x7f05e809a660 con 0x7f05f015c450 2024-09-10T22:41:05.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.099+0000 7f05f4fdc640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f05d0077f70 0x7f05d007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:05.101 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.099+0000 7f05f4fdc640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f05d0077f70 0x7f05d007a430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05e0033000 tx=0x7f05e002d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:05.103 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.103+0000 7f05ed7fa640 1 -- 172.21.15.66:0/872238831 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f05e8062be0 con 0x7f05f015c450 2024-09-10T22:41:05.289 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.288+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f05f00d4ea0 con 0x7f05f015c450 2024-09-10T22:41:05.973 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.972+0000 7f05ed7fa640 1 -- 172.21.15.66:0/872238831 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) ==== 169+0+0 (secure 0 0 0) 0x7f05e8067610 con 0x7f05f015c450 2024-09-10T22:41:05.973 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2024-09-10T22:41:05.976 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.975+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f05d0077f70 msgr2=0x7f05d007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:05.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.975+0000 7f05f7240640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f05d0077f70 0x7f05d007a430 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05e0033000 tx=0x7f05e002d040 comp rx=0 tx=0).stop 2024-09-10T22:41:05.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.975+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05f015c450 msgr2=0x7f05f015c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:05.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.975+0000 7f05f7240640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05f015c450 0x7f05f015c880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f05f00d1440 tx=0x7f05e8011d50 comp rx=0 tx=0).stop 2024-09-10T22:41:05.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 shutdown_connections 2024-09-10T22:41:05.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f05d0077f70 0x7f05d007a430 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05f015c450 0x7f05f015c880 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05f00b3630 0x7f05f00b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 --2- 172.21.15.66:0/872238831 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f05dc003a00 0x7f05f015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 >> 172.21.15.66:0/872238831 conn(0x7f05f008dac0 msgr2=0x7f05f006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 shutdown_connections 2024-09-10T22:41:05.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:05.976+0000 7f05f7240640 1 -- 172.21.15.66:0/872238831 wait complete. 2024-09-10T22:41:05.991 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:41:05.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: chdir = "" 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: auth supported = cephx 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: filestore xattr use omap = true 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: mon clock drift allowed = 1.000 2024-09-10T22:41:06.038 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: osd crush chooseleaf type = 0 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: auth debug = true 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on old message = true 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on bug = true 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: mon pg warn max object skew = 0 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:41:06.039 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default size = 2 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow primary affinity = true 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd allow pg remap = true 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on no sortbitwise = false 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon warn on too few osds = false 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:41:06.040 INFO:teuthology.orchestra.run.smithi066.stdout: mon_allow_pool_size_one = true 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: osd default data pool replay window = 5 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: mon allow pool delete = true 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: mon cluster log file level = debug 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: debug asserts on shutdown = true 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: mon health detail to clog = false 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:41:06.041 INFO:teuthology.orchestra.run.smithi066.stdout: ms die on skipped message = False 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout:[osd] 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: osd journal size = 100 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub load threshold = 5.0 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: osd scrub max interval = 600 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: osd recover clone overlap = true 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: osd recovery max chunk = 1048576 2024-09-10T22:41:06.042 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug shutdown = true 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug op order = true 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug verify stray on activate = true 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug trim objects = true 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd open classes on start = true 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug pg log writeout = true 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.043 INFO:teuthology.orchestra.run.smithi066.stdout: osd map max advance = 10 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: journal zero on create = true 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: filestore apply finisher threads = 3 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: bdev debug aio = true 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: osd debug misdirected ops = true 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: bdev async discard = True 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: bdev enable discard = True 2024-09-10T22:41:06.044 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore allocator = bitmap 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore block size = 96636764160 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: bluestore fsck on mount = True 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluefs = 1/20 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: debug bluestore = 1/20 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: debug osd = 20 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: debug rocksdb = 4/10 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd full ratio = 0.9 2024-09-10T22:41:06.045 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: osd objectstore = bluestore 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout:[mgr] 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: debug mgr = 20 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: mgr/telemetry/nag = false 2024-09-10T22:41:06.046 INFO:teuthology.orchestra.run.smithi066.stdout: client mount timeout = 30 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: debug client = 20 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout:[mon] 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: debug ms = 1 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: debug mon = 20 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: debug paxos = 20 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: debug auth = 20 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: mon data avail warn = 5 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: mon mgr mkfs grace = 240 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:41:06.047 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd reporter subtree level = osd 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: mon osd prime pg temp = true 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:41:06.048 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: # 1m isn't quite enough 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: mon down mkfs grace = 300 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout:[client] 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: rgw cache enabled = true 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable ops log = true 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: rgw enable usage log = true 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:41:06.049 INFO:teuthology.orchestra.run.smithi066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout: pid file = "" 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.a] 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.b] 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mon.c] 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a] 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c] 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b] 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d] 2024-09-10T22:41:06.050 INFO:teuthology.orchestra.run.smithi066.stdout:[mds] 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-10T22:41:06.051 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T22:41:06.052 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-10T22:41:06.053 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.054 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.66,172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790]" 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on skipped message = False 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 7ef137cd-a1ce-4819-b27d-b71268612f85 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-10T22:41:06.055 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-10T22:41:06.056 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-10T22:41:06.057 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: bdev async discard = True 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: bdev enable discard = True 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore allocator = bitmap 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-10T22:41:06.058 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:41:06.059 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: client mount timeout = 30 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: debug client = 20 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-10T22:41:06.060 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T22:41:06.061 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-10T22:41:06.062 INFO:teuthology.orchestra.run.smithi094.stdout: mon down mkfs grace = 300 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = "" 2024-09-10T22:41:06.063 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-10T22:41:06.064 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-10T22:41:06.064 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-10T22:41:06.064 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a] 2024-09-10T22:41:06.064 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c] 2024-09-10T22:41:06.064 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b] 2024-09-10T22:41:06.064 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d] 2024-09-10T22:41:06.064 INFO:teuthology.orchestra.run.smithi094.stdout:[mds] 2024-09-10T22:41:06.066 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T22:41:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 -- 172.21.15.66:0/2666529876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380aee40 msgr2=0x7fbb380cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 --2- 172.21.15.66:0/2666529876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380aee40 0x7fbb380cec20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb3002aa10 comp rx=0 tx=0).stop 2024-09-10T22:41:06.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 -- 172.21.15.66:0/2666529876 shutdown_connections 2024-09-10T22:41:06.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 --2- 172.21.15.66:0/2666529876 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380aee40 0x7fbb380cec20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 --2- 172.21.15.66:0/2666529876 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb380ce840 0x7fbb380d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 -- 172.21.15.66:0/2666529876 >> 172.21.15.66:0/2666529876 conn(0x7fbb380d1e30 msgr2=0x7fbb380d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:06.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 -- 172.21.15.66:0/2666529876 shutdown_connections 2024-09-10T22:41:06.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.211+0000 7fbb40433640 1 -- 172.21.15.66:0/2666529876 wait complete. 2024-09-10T22:41:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.212+0000 7fbb40433640 1 Processor -- start 2024-09-10T22:41:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.212+0000 7fbb40433640 1 -- start start 2024-09-10T22:41:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb40433640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb380ae9e0 0x7fbb380b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb40433640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb380aee40 0x7fbb380b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb40433640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380ce840 0x7fbb38153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb40433640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380cf160 con 0x7fbb380ce840 2024-09-10T22:41:06.216 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb40433640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380cdbc0 con 0x7fbb380ae9e0 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb40433640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbb380cf2e0 con 0x7fbb380aee40 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380ce840 0x7fbb38153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380ce840 0x7fbb38153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36756/0 (socket says 172.21.15.66:36756) 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 -- 172.21.15.66:0/4155836207 learned_addr learned my addr 172.21.15.66:0/4155836207 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e1cf640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb380ae9e0 0x7fbb380b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3d9ce640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb380aee40 0x7fbb380b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 -- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb380aee40 msgr2=0x7fbb380b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb380aee40 0x7fbb380b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 -- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb380ae9e0 msgr2=0x7fbb380b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb380ae9e0 0x7fbb380b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.217 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 -- 172.21.15.66:0/4155836207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380b44f0 con 0x7fbb380ce840 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb3e9d0640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380ce840 0x7fbb38153470 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb3002b000 tx=0x7fbb30003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb2effd640 1 -- 172.21.15.66:0/4155836207 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30046070 con 0x7fbb380ce840 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.213+0000 7fbb2effd640 1 -- 172.21.15.66:0/4155836207 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb30004250 con 0x7fbb380ce840 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.214+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb381539b0 con 0x7fbb380ce840 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.214+0000 7fbb2effd640 1 -- 172.21.15.66:0/4155836207 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30033050 con 0x7fbb380ce840 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.214+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb380cac00 con 0x7fbb380ce840 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.214+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb380cec20 con 0x7fbb380ce840 2024-09-10T22:41:06.218 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.215+0000 7fbb2effd640 1 -- 172.21.15.66:0/4155836207 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbb30004660 con 0x7fbb380ce840 2024-09-10T22:41:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.216+0000 7fbb2effd640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbb1c077fa0 0x7fbb1c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.216+0000 7fbb2effd640 1 -- 172.21.15.66:0/4155836207 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fbb300baba0 con 0x7fbb380ce840 2024-09-10T22:41:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.216+0000 7fbb3e1cf640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbb1c077fa0 0x7fbb1c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.219 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.216+0000 7fbb3e1cf640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbb1c077fa0 0x7fbb1c07a460 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbb34004630 tx=0x7fbb3400b2e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:06.221 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.220+0000 7fbb2effd640 1 -- 172.21.15.66:0/4155836207 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbb30083380 con 0x7fbb380ce840 2024-09-10T22:41:06.408 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.407+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fbb380b7db0 con 0x7fbb380ce840 2024-09-10T22:41:06.411 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.410+0000 7fbb2effd640 1 -- 172.21.15.66:0/4155836207 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v31) ==== 71+0+0 (secure 0 0 0) 0x7fbb3008c0f0 con 0x7fbb380ce840 2024-09-10T22:41:06.413 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbb1c077fa0 msgr2=0x7fbb1c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbb1c077fa0 0x7fbb1c07a460 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbb34004630 tx=0x7fbb3400b2e0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380ce840 msgr2=0x7fbb38153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380ce840 0x7fbb38153470 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb3002b000 tx=0x7fbb30003f10 comp rx=0 tx=0).stop 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 shutdown_connections 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbb1c077fa0 0x7fbb1c07a460 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbb380ce840 0x7fbb38153470 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbb380aee40 0x7fbb380b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 --2- 172.21.15.66:0/4155836207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb380ae9e0 0x7fbb380b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 >> 172.21.15.66:0/4155836207 conn(0x7fbb380d1e30 msgr2=0x7fbb380b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 shutdown_connections 2024-09-10T22:41:06.414 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.412+0000 7fbb40433640 1 -- 172.21.15.66:0/4155836207 wait complete. 2024-09-10T22:41:06.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-09-10T22:41:06.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 -- 172.21.15.66:0/1677015280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048152a40 msgr2=0x7fc048152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.530 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/1677015280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048152a40 0x7fc048152e20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc038005fd0 tx=0x7fc03802a4b0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 -- 172.21.15.66:0/1677015280 shutdown_connections 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/1677015280 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0480ca4b0 0x7fc0480ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/1677015280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048152a40 0x7fc048152e20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 -- 172.21.15.66:0/1677015280 >> 172.21.15.66:0/1677015280 conn(0x7fc0480b1950 msgr2=0x7fc0480b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 -- 172.21.15.66:0/1677015280 shutdown_connections 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.529+0000 7fc04e8a9640 1 -- 172.21.15.66:0/1677015280 wait complete. 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.530+0000 7fc04e8a9640 1 Processor -- start 2024-09-10T22:41:06.531 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.530+0000 7fc04e8a9640 1 -- start start 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc04e8a9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048072810 0x7fc0481f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc04e8a9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0480ca4b0 0x7fc0480cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc04e8a9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048152a40 0x7fc0480ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc04e8a9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fc048070820 con 0x7fc048072810 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc04e8a9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc0480706a0 con 0x7fc048152a40 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc04e8a9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fc0480709a0 con 0x7fc0480ca4b0 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc047fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048072810 0x7fc0481f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc047fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048072810 0x7fc0481f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36780/0 (socket says 172.21.15.66:36780) 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.531+0000 7fc047fff640 1 -- 172.21.15.66:0/2631298260 learned_addr learned my addr 172.21.15.66:0/2631298260 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc0477fe640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0480ca4b0 0x7fc0480cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.534 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc047fff640 1 -- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0480ca4b0 msgr2=0x7fc0480cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc047fff640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0480ca4b0 0x7fc0480cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc04ce46640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048152a40 0x7fc0480ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc047fff640 1 -- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048152a40 msgr2=0x7fc0480ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc047fff640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048152a40 0x7fc0480ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc047fff640 1 -- 172.21.15.66:0/2631298260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0480cf730 con 0x7fc048072810 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc0477fe640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0480ca4b0 0x7fc0480cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc047fff640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048072810 0x7fc0481f53d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc038034040 tx=0x7fc038003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc044ff9640 1 -- 172.21.15.66:0/2631298260 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03803e030 con 0x7fc048072810 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc044ff9640 1 -- 172.21.15.66:0/2631298260 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc038003e00 con 0x7fc048072810 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0480cdad0 con 0x7fc048072810 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc044ff9640 1 -- 172.21.15.66:0/2631298260 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038035040 con 0x7fc048072810 2024-09-10T22:41:06.535 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.532+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0480ce070 con 0x7fc048072810 2024-09-10T22:41:06.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.533+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0480ca900 con 0x7fc048072810 2024-09-10T22:41:06.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.534+0000 7fc044ff9640 1 -- 172.21.15.66:0/2631298260 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc038004a00 con 0x7fc048072810 2024-09-10T22:41:06.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.534+0000 7fc044ff9640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc034077f70 0x7fc03407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:06.536 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.534+0000 7fc044ff9640 1 -- 172.21.15.66:0/2631298260 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fc0380ba4c0 con 0x7fc048072810 2024-09-10T22:41:06.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.535+0000 7fc0477fe640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc034077f70 0x7fc03407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:06.537 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.535+0000 7fc0477fe640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc034077f70 0x7fc03407a430 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc04814e3f0 tx=0x7fc03c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:06.539 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.538+0000 7fc044ff9640 1 -- 172.21.15.66:0/2631298260 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc038083260 con 0x7fc048072810 2024-09-10T22:41:06.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.798+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) -- 0x7fc0480b27d0 con 0x7fc048072810 2024-09-10T22:41:06.974 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.973+0000 7fc044ff9640 1 -- 172.21.15.66:0/2631298260 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v250) ==== 143+0+0 (secure 0 0 0) 0x7fc038082b60 con 0x7fc048072810 2024-09-10T22:41:06.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc034077f70 msgr2=0x7fc03407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc034077f70 0x7fc03407a430 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc04814e3f0 tx=0x7fc03c009040 comp rx=0 tx=0).stop 2024-09-10T22:41:06.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048072810 msgr2=0x7fc0481f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:06.977 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048072810 0x7fc0481f53d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc038034040 tx=0x7fc038003910 comp rx=0 tx=0).stop 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 shutdown_connections 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fc034077f70 0x7fc03407a430 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc048152a40 0x7fc0480ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fc0480ca4b0 0x7fc0480cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 --2- 172.21.15.66:0/2631298260 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc048072810 0x7fc0481f53d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 >> 172.21.15.66:0/2631298260 conn(0x7fc0480b1950 msgr2=0x7fc0480b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 shutdown_connections 2024-09-10T22:41:06.978 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:06.976+0000 7fc04e8a9640 1 -- 172.21.15.66:0/2631298260 wait complete. 2024-09-10T22:41:06.988 INFO:tasks.cephfs_test_runner:Opening a file should increment omap_total_updates by 1. ... ok 2024-09-10T22:41:06.988 INFO:tasks.cephfs_test_runner: 2024-09-10T22:41:06.988 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T22:41:06.989 INFO:tasks.cephfs_test_runner:Ran 2 tests in 340.000s 2024-09-10T22:41:06.989 INFO:tasks.cephfs_test_runner: 2024-09-10T22:41:06.989 INFO:tasks.cephfs_test_runner:OK 2024-09-10T22:41:06.989 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T22:41:06.998 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T22:41:07.013 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T22:41:07.013 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:41:07.013 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:07.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=930 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:41:07.030 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T22:41:07.030 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:07.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056004,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=930 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T22:41:07.086 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:41:07.086 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:07.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:41:07.105 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-10T22:41:07.105 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:41:07.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=770 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T22:41:07.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.163 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:41:07.179 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T22:41:07.180 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:41:07.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T22:41:07.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.246 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:41:07.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.312 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-09-10T22:41:07.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.410 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2024-09-10T22:41:07.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2024-09-10T22:41:07.494 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2024-09-10T22:41:07.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:41:07.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.496 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:41:07.515 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T22:41:07.515 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T22:41:07.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T22:41:07.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T22:41:07.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.655 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T22:41:07.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.748 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-09-10T22:41:07.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:41:07.764 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-09-10T22:41:07.834 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-09-10T22:41:07.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:41:07.835 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T22:41:07.844 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T22:41:07.844 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:41:07.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.950+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/1623226445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a98070210 msgr2=0x7f5a980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:07.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.950+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/1623226445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a98070210 0x7f5a980705f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009830 tx=0x7f5a8802a5c0 comp rx=0 tx=0).stop 2024-09-10T22:41:07.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.950+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/1623226445 shutdown_connections 2024-09-10T22:41:07.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.950+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/1623226445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a98070210 0x7f5a980705f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:07.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.950+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/1623226445 >> 172.21.15.66:0/1623226445 conn(0x7f5a980b1950 msgr2=0x7f5a980b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:07.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.950+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/1623226445 shutdown_connections 2024-09-10T22:41:07.952 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.950+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/1623226445 wait complete. 2024-09-10T22:41:07.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.952+0000 7f5a9e5f3640 1 Processor -- start 2024-09-10T22:41:07.953 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.952+0000 7f5a9e5f3640 1 -- start start 2024-09-10T22:41:07.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9e5f3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a98070210 0x7f5a981f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:07.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9e5f3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a98073040 0x7f5a980c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:07.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9e5f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a980ca4b0 0x7f5a98151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:07.955 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9e5f3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981521b0 con 0x7f5a98070210 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9e5f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9806ef80 con 0x7f5a980ca4b0 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9e5f3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5a980cb1e0 con 0x7f5a98073040 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a98070210 0x7f5a981f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a98070210 0x7f5a981f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36796/0 (socket says 172.21.15.66:36796) 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 -- 172.21.15.66:0/459815041 learned_addr learned my addr 172.21.15.66:0/459815041 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9cb90640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a980ca4b0 0x7f5a98151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a977fe640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a98073040 0x7f5a980c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 -- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a98073040 msgr2=0x7f5a980c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a98073040 0x7f5a980c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 -- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a980ca4b0 msgr2=0x7f5a98151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a980ca4b0 0x7f5a98151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 -- 172.21.15.66:0/459815041 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a980c7140 con 0x7f5a98070210 2024-09-10T22:41:07.956 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a9cb90640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a980ca4b0 0x7f5a98151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a977fe640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a98073040 0x7f5a980c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a97fff640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a98070210 0x7f5a981f3570 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009950 tx=0x7f5a88032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a94ff9640 1 -- 172.21.15.66:0/459815041 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88040070 con 0x7f5a98070210 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.953+0000 7f5a94ff9640 1 -- 172.21.15.66:0/459815041 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a88005310 con 0x7f5a98070210 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.954+0000 7f5a94ff9640 1 -- 172.21.15.66:0/459815041 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880047a0 con 0x7f5a98070210 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.954+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a980c7420 con 0x7f5a98070210 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.954+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a980c79e0 con 0x7f5a98070210 2024-09-10T22:41:07.957 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.955+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a980cb690 con 0x7f5a98070210 2024-09-10T22:41:07.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.955+0000 7f5a94ff9640 1 -- 172.21.15.66:0/459815041 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5a88004940 con 0x7f5a98070210 2024-09-10T22:41:07.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.956+0000 7f5a94ff9640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a84077f70 0x7f5a8407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:07.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.956+0000 7f5a94ff9640 1 -- 172.21.15.66:0/459815041 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f5a880baf10 con 0x7f5a98070210 2024-09-10T22:41:07.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.956+0000 7f5a977fe640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a84077f70 0x7f5a8407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:07.958 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.956+0000 7f5a977fe640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a84077f70 0x7f5a8407a430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5a9815abb0 tx=0x7f5a8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:07.961 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:07.960+0000 7f5a94ff9640 1 -- 172.21.15.66:0/459815041 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5a88083cb0 con 0x7f5a98070210 2024-09-10T22:41:08.145 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.144+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a98074ad0 con 0x7f5a84077f70 2024-09-10T22:41:08.146 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.145+0000 7f5a94ff9640 1 -- 172.21.15.66:0/459815041 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13417 (secure 0 0 0) 0x7f5a98074ad0 con 0x7f5a84077f70 2024-09-10T22:41:08.146 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:08.146 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:41:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a84077f70 msgr2=0x7f5a8407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.148 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a84077f70 0x7f5a8407a430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5a9815abb0 tx=0x7f5a8c009040 comp rx=0 tx=0).stop 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a98070210 msgr2=0x7f5a981f3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a98070210 0x7f5a981f3570 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009950 tx=0x7f5a88032b90 comp rx=0 tx=0).stop 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 shutdown_connections 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5a84077f70 0x7f5a8407a430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a980ca4b0 0x7f5a98151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5a98073040 0x7f5a980c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 --2- 172.21.15.66:0/459815041 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5a98070210 0x7f5a981f3570 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 >> 172.21.15.66:0/459815041 conn(0x7f5a980b1950 msgr2=0x7f5a980ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 shutdown_connections 2024-09-10T22:41:08.149 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.147+0000 7f5a9e5f3640 1 -- 172.21.15.66:0/459815041 wait complete. 2024-09-10T22:41:08.159 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T22:41:06.685606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243124,"kb_used_data":6108,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731596,"statfs":{"total":773094113280,"available":772845154304,"internally_reserved":0,"allocated":6254592,"data_stored":3286687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7290,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-74,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001542"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T22:39:44.839303+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:39:44.839303+0000","last_peered":"2024-09-10T22:39:44.839303+0000","last_clean":"2024-09-10T22:39:44.839303+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:39:44.839303+0000","last_undegraded":"2024-09-10T22:39:44.839303+0000","last_fullsized":"2024-09-10T22:39:44.839303+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1216512,"data_stored":849545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509517,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":630784,"data_stored":259766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":622592,"data_stored":259157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":634880,"data_stored":265572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509518,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":849496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542223,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":265950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542224,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":271819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":634880,"data_stored":265382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:41:08.159 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:41:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.263+0000 7f79558ed640 1 -- 172.21.15.66:0/659712404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380051c0 msgr2=0x7f793800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.263+0000 7f79558ed640 1 --2- 172.21.15.66:0/659712404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380051c0 0x7f793800b9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7948009ed0 tx=0x7f7938005830 comp rx=0 tx=0).stop 2024-09-10T22:41:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.263+0000 7f79558ed640 1 -- 172.21.15.66:0/659712404 shutdown_connections 2024-09-10T22:41:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.263+0000 7f79558ed640 1 --2- 172.21.15.66:0/659712404 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79380051c0 0x7f793800b9c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.265 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.264+0000 7f79558ed640 1 -- 172.21.15.66:0/659712404 >> 172.21.15.66:0/659712404 conn(0x7f79500b1950 msgr2=0x7f79500b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.264+0000 7f79558ed640 1 -- 172.21.15.66:0/659712404 shutdown_connections 2024-09-10T22:41:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.264+0000 7f79558ed640 1 -- 172.21.15.66:0/659712404 wait complete. 2024-09-10T22:41:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.265+0000 7f79558ed640 1 Processor -- start 2024-09-10T22:41:08.266 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.265+0000 7f79558ed640 1 -- start start 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f79558ed640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79380051c0 0x7f79501f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f79558ed640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79501f5bb0 0x7f79501450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f79558ed640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79500c4de0 0x7f79501455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f79558ed640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f795015c870 con 0x7f79500c4de0 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f79558ed640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f795015c6f0 con 0x7f79501f5bb0 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f79558ed640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f79500ca4b0 con 0x7f79380051c0 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79500c4de0 0x7f79501455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79500c4de0 0x7f79501455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36822/0 (socket says 172.21.15.66:36822) 2024-09-10T22:41:08.268 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 -- 172.21.15.66:0/1675106032 learned_addr learned my addr 172.21.15.66:0/1675106032 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794effd640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79380051c0 0x7f79501f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794e7fc640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79501f5bb0 0x7f79501450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 -- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79380051c0 msgr2=0x7f79501f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79380051c0 0x7f79501f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 -- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79501f5bb0 msgr2=0x7f79501450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79501f5bb0 0x7f79501450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 -- 172.21.15.66:0/1675106032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79500c5280 con 0x7f79500c4de0 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794effd640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79380051c0 0x7f79501f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f794f7fe640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79500c4de0 0x7f79501455f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7948009e20 tx=0x7f7948034c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.266+0000 7f7937fff640 1 -- 172.21.15.66:0/1675106032 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948041070 con 0x7f79500c4de0 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.267+0000 7f7937fff640 1 -- 172.21.15.66:0/1675106032 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79480325f0 con 0x7f79500c4de0 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.267+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79500c5560 con 0x7f79500c4de0 2024-09-10T22:41:08.269 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.267+0000 7f7937fff640 1 -- 172.21.15.66:0/1675106032 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948009ad0 con 0x7f79500c4de0 2024-09-10T22:41:08.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.267+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79500c5b20 con 0x7f79500c4de0 2024-09-10T22:41:08.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.268+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79500cbb00 con 0x7f79500c4de0 2024-09-10T22:41:08.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.268+0000 7f7937fff640 1 -- 172.21.15.66:0/1675106032 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7948032a40 con 0x7f79500c4de0 2024-09-10T22:41:08.270 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.269+0000 7f7937fff640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f793c077f70 0x7f793c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.269+0000 7f7937fff640 1 -- 172.21.15.66:0/1675106032 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f79480ba1c0 con 0x7f79500c4de0 2024-09-10T22:41:08.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.269+0000 7f794effd640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f793c077f70 0x7f793c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.271 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.269+0000 7f794effd640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f793c077f70 0x7f793c07a430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7940006fd0 tx=0x7f7940006b10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:08.274 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.273+0000 7f7937fff640 1 -- 172.21.15.66:0/1675106032 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7948082800 con 0x7f79500c4de0 2024-09-10T22:41:08.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.453+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f79500b27d0 con 0x7f793c077f70 2024-09-10T22:41:08.454 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.454+0000 7f7937fff640 1 -- 172.21.15.66:0/1675106032 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13417 (secure 0 0 0) 0x7f79500b27d0 con 0x7f793c077f70 2024-09-10T22:41:08.455 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:08.455 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:41:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f793c077f70 msgr2=0x7f793c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f793c077f70 0x7f793c07a430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7940006fd0 tx=0x7f7940006b10 comp rx=0 tx=0).stop 2024-09-10T22:41:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79500c4de0 msgr2=0x7f79501455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79500c4de0 0x7f79501455f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7948009e20 tx=0x7f7948034c20 comp rx=0 tx=0).stop 2024-09-10T22:41:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 shutdown_connections 2024-09-10T22:41:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f793c077f70 0x7f793c07a430 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f79500c4de0 0x7f79501455f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79501f5bb0 0x7f79501450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 --2- 172.21.15.66:0/1675106032 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f79380051c0 0x7f79501f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 >> 172.21.15.66:0/1675106032 conn(0x7f79500b1950 msgr2=0x7f79500b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 shutdown_connections 2024-09-10T22:41:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.456+0000 7f79558ed640 1 -- 172.21.15.66:0/1675106032 wait complete. 2024-09-10T22:41:08.468 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-10T22:41:06.685606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243124,"kb_used_data":6108,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731596,"statfs":{"total":773094113280,"available":772845154304,"internally_reserved":0,"allocated":6254592,"data_stored":3286687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7290,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-74,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001542"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":169,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-10T22:39:44.839303+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:39:44.839303+0000","last_peered":"2024-09-10T22:39:44.839303+0000","last_clean":"2024-09-10T22:39:44.839303+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:39:44.839303+0000","last_undegraded":"2024-09-10T22:39:44.839303+0000","last_fullsized":"2024-09-10T22:39:44.839303+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509517,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1216512,"data_stored":849545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509517,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":630784,"data_stored":259766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":622592,"data_stored":259157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":634880,"data_stored":265572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509518,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":849496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542223,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":265950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542224,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":271819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":634880,"data_stored":265382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:41:08.468 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T22:41:08.468 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:41:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/187612012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd580 msgr2=0x7f6a840adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/187612012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd580 0x7f6a840adda0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6a840d0e20 tx=0x7f6a7802a530 comp rx=0 tx=0).stop 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/187612012 shutdown_connections 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/187612012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd580 0x7f6a840adda0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/187612012 >> v2:172.21.15.94:3300/0 conn(0x7f6a840cd1a0 0x7f6a840cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/187612012 >> 172.21.15.66:0/187612012 conn(0x7f6a840cd9e0 msgr2=0x7f6a840cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/187612012 shutdown_connections 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.572+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/187612012 wait complete. 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.573+0000 7f6a8b1f3640 1 Processor -- start 2024-09-10T22:41:08.574 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.573+0000 7f6a8b1f3640 1 -- start start 2024-09-10T22:41:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a8b1f3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a840ae2e0 0x7f6a840b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a8b1f3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd1a0 0x7f6a84156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.576 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a8b1f3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a840cd580 0x7f6a84142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a8b1f3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6a840afd80 con 0x7f6a840cd1a0 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a8b1f3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a840afc00 con 0x7f6a840ae2e0 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a8b1f3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6a840aff00 con 0x7f6a840cd580 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a83fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd1a0 0x7f6a84156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a83fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd1a0 0x7f6a84156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36848/0 (socket says 172.21.15.66:36848) 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a83fff640 1 -- 172.21.15.66:0/4191993854 learned_addr learned my addr 172.21.15.66:0/4191993854 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.574+0000 7f6a88f8f640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a840ae2e0 0x7f6a840b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a89790640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a840cd580 0x7f6a84142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a83fff640 1 -- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a840cd580 msgr2=0x7f6a84142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a83fff640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a840cd580 0x7f6a84142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a83fff640 1 -- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a840ae2e0 msgr2=0x7f6a840b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a83fff640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a840ae2e0 0x7f6a840b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.577 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a83fff640 1 -- 172.21.15.66:0/4191993854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a840b2700 con 0x7f6a840cd1a0 2024-09-10T22:41:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a88f8f640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a840ae2e0 0x7f6a840b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:41:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a83fff640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd1a0 0x7f6a84156940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009780 tx=0x7f6a78033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a817fa640 1 -- 172.21.15.66:0/4191993854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78031d60 con 0x7f6a840cd1a0 2024-09-10T22:41:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a817fa640 1 -- 172.21.15.66:0/4191993854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a78033e10 con 0x7f6a840cd1a0 2024-09-10T22:41:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a840b29e0 con 0x7f6a840cd1a0 2024-09-10T22:41:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a817fa640 1 -- 172.21.15.66:0/4191993854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78005260 con 0x7f6a840cd1a0 2024-09-10T22:41:08.578 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.575+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a840b2fa0 con 0x7f6a840cd1a0 2024-09-10T22:41:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.576+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a840b03b0 con 0x7f6a840cd1a0 2024-09-10T22:41:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.577+0000 7f6a817fa640 1 -- 172.21.15.66:0/4191993854 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6a78005400 con 0x7f6a840cd1a0 2024-09-10T22:41:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.577+0000 7f6a817fa640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6a70077fa0 0x7f6a7007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.577+0000 7f6a817fa640 1 -- 172.21.15.66:0/4191993854 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f6a780babe0 con 0x7f6a840cd1a0 2024-09-10T22:41:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.578+0000 7f6a88f8f640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6a70077fa0 0x7f6a7007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.579 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.578+0000 7f6a88f8f640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6a70077fa0 0x7f6a7007a460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a740058c0 tx=0x7f6a74005850 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:08.582 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.581+0000 7f6a817fa640 1 -- 172.21.15.66:0/4191993854 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6a7803f030 con 0x7f6a840cd1a0 2024-09-10T22:41:08.764 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.763+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a840746b0 con 0x7f6a70077fa0 2024-09-10T22:41:08.765 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.764+0000 7f6a817fa640 1 -- 172.21.15.66:0/4191993854 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13457 (secure 0 0 0) 0x7f6a840746b0 con 0x7f6a70077fa0 2024-09-10T22:41:08.765 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:08.766 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6a70077fa0 msgr2=0x7f6a7007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6a70077fa0 0x7f6a7007a460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a740058c0 tx=0x7f6a74005850 comp rx=0 tx=0).stop 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd1a0 msgr2=0x7f6a84156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd1a0 0x7f6a84156940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009780 tx=0x7f6a78033c20 comp rx=0 tx=0).stop 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 shutdown_connections 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6a70077fa0 0x7f6a7007a460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6a840cd580 0x7f6a84142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.768 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6a840cd1a0 0x7f6a84156940 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 --2- 172.21.15.66:0/4191993854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a840ae2e0 0x7f6a840b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 >> 172.21.15.66:0/4191993854 conn(0x7f6a840cd9e0 msgr2=0x7f6a840cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:08.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 shutdown_connections 2024-09-10T22:41:08.769 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.767+0000 7f6a8b1f3640 1 -- 172.21.15.66:0/4191993854 wait complete. 2024-09-10T22:41:08.779 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-10T22:41:08.685985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":243280,"kb_used_data":6244,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731440,"statfs":{"total":773094113280,"available":772844994560,"internally_reserved":0,"allocated":6393856,"data_stored":3352566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7290,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-74,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001442"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":175,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T22:41:05.976874+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:41:05.976874+0000","last_peered":"2024-09-10T22:41:05.976874+0000","last_clean":"2024-09-10T22:41:05.976874+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:41:05.976874+0000","last_undegraded":"2024-09-10T22:41:05.976874+0000","last_fullsized":"2024-09-10T22:41:05.976874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":867512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509518,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":271466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":647168,"data_stored":271459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509519,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1245184,"data_stored":861834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542223,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":265950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542224,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":271819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":634880,"data_stored":265382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:41:08.779 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:41:08.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.883+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2623030403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 msgr2=0x7fe1f415f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.884 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.883+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2623030403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f415f000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec005fd0 tx=0x7fe1ec02a9d0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.883+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2623030403 shutdown_connections 2024-09-10T22:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.883+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2623030403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f415f000 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.883+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2623030403 >> 172.21.15.66:0/2623030403 conn(0x7fe1f40d1e30 msgr2=0x7fe1f40d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.883+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2623030403 shutdown_connections 2024-09-10T22:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.883+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2623030403 wait complete. 2024-09-10T22:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.884+0000 7fe1f9dd3640 1 Processor -- start 2024-09-10T22:41:08.885 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f9dd3640 1 -- start start 2024-09-10T22:41:08.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f9dd3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.887 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f9dd3640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1f40ce830 0x7fe1f41f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f9dd3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f40d2950 0x7fe1f41f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f9dd3640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f40ced90 con 0x7fe1f40cdb60 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f9dd3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f40cec10 con 0x7fe1f40d2950 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f9dd3640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f40cef10 con 0x7fe1f40ce830 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f41509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36860/0 (socket says 172.21.15.66:36860) 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.885+0000 7fe1f37fe640 1 -- 172.21.15.66:0/2641683268 learned_addr learned my addr 172.21.15.66:0/2641683268 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f2ffd640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1f40ce830 0x7fe1f41f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f3fff640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f40d2950 0x7fe1f41f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f37fe640 1 -- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1f40ce830 msgr2=0x7fe1f41f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f37fe640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1f40ce830 0x7fe1f41f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.888 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f37fe640 1 -- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f40d2950 msgr2=0x7fe1f41f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f37fe640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f40d2950 0x7fe1f41f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f37fe640 1 -- 172.21.15.66:0/2641683268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f40c83c0 con 0x7fe1f40cdb60 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f37fe640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f41509d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c410 tx=0x7fe1e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1dbfff640 1 -- 172.21.15.66:0/2641683268 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4017070 con 0x7fe1f40cdb60 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1dbfff640 1 -- 172.21.15.66:0/2641683268 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e40035f0 con 0x7fe1f40cdb60 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f40c86a0 con 0x7fe1f40cdb60 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1dbfff640 1 -- 172.21.15.66:0/2641683268 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4003a80 con 0x7fe1f40cdb60 2024-09-10T22:41:08.889 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.886+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f40c9db0 con 0x7fe1f40cdb60 2024-09-10T22:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.887+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f415f000 con 0x7fe1f40cdb60 2024-09-10T22:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.888+0000 7fe1dbfff640 1 -- 172.21.15.66:0/2641683268 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe1e4003c20 con 0x7fe1f40cdb60 2024-09-10T22:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.888+0000 7fe1dbfff640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1e0077f70 0x7fe1e007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.888+0000 7fe1dbfff640 1 -- 172.21.15.66:0/2641683268 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fe1e4099060 con 0x7fe1f40cdb60 2024-09-10T22:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.889+0000 7fe1f2ffd640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1e0077f70 0x7fe1e007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:08.890 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.889+0000 7fe1f2ffd640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1e0077f70 0x7fe1e007a430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4151d90 tx=0x7fe1e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:08.893 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:08.893+0000 7fe1dbfff640 1 -- 172.21.15.66:0/2641683268 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe1e40617b0 con 0x7fe1f40cdb60 2024-09-10T22:41:09.079 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.078+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1f40d1160 con 0x7fe1f40cdb60 2024-09-10T22:41:09.080 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.079+0000 7fe1dbfff640 1 -- 172.21.15.66:0/2641683268 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+13136 (secure 0 0 0) 0x7fe1e40661e0 con 0x7fe1f40cdb60 2024-09-10T22:41:09.080 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:09.080 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"fsid":"7ef137cd-a1ce-4819-b27d-b71268612f85","created":"2024-09-10T22:34:38.227379+0000","modified":"2024-09-10T22:41:05.966664+0000","last_up_change":"2024-09-10T22:34:50.792160+0000","last_in_change":"2024-09-10T22:34:41.329138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T22:34:50.607631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"418ab0eb-5cba-46fb-a4ce-f29a4be053b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6809","nonce":1933221850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6811","nonce":1933221850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6815","nonce":1933221850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":1933221850},{"type":"v1","addr":"172.21.15.66:6813","nonce":1933221850}]},"public_addr":"172.21.15.66:6809/1933221850","cluster_addr":"172.21.15.66:6811/1933221850","heartbeat_back_addr":"172.21.15.66:6815/1933221850","heartbeat_front_addr":"172.21.15.66:6813/1933221850","state":["exists","up"]},{"osd":1,"uuid":"35dec180-b760-4a24-93ab-3b1573a03614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6801","nonce":1998688775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6803","nonce":1998688775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6807","nonce":1998688775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1998688775},{"type":"v1","addr":"172.21.15.66:6805","nonce":1998688775}]},"public_addr":"172.21.15.66:6801/1998688775","cluster_addr":"172.21.15.66:6803/1998688775","heartbeat_back_addr":"172.21.15.66:6807/1998688775","heartbeat_front_addr":"172.21.15.66:6805/1998688775","state":["exists","up"]},{"osd":2,"uuid":"28d55995-27ac-4899-9a76-11ef96760d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6817","nonce":3996225872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6819","nonce":3996225872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6823","nonce":3996225872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":3996225872},{"type":"v1","addr":"172.21.15.66:6821","nonce":3996225872}]},"public_addr":"172.21.15.66:6817/3996225872","cluster_addr":"172.21.15.66:6819/3996225872","heartbeat_back_addr":"172.21.15.66:6823/3996225872","heartbeat_front_addr":"172.21.15.66:6821/3996225872","state":["exists","up"]},{"osd":3,"uuid":"7cb4f0ea-0296-4f5e-959e-33abec73e116","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6825","nonce":2075348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6827","nonce":2075348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6831","nonce":2075348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":2075348943},{"type":"v1","addr":"172.21.15.66:6829","nonce":2075348943}]},"public_addr":"172.21.15.66:6825/2075348943","cluster_addr":"172.21.15.66:6827/2075348943","heartbeat_back_addr":"172.21.15.66:6831/2075348943","heartbeat_front_addr":"172.21.15.66:6829/2075348943","state":["exists","up"]},{"osd":4,"uuid":"a9663308-591b-44e9-856b-c0666fd661b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6819","nonce":3674153549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6821","nonce":3674153549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6825","nonce":3674153549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":3674153549},{"type":"v1","addr":"172.21.15.94:6823","nonce":3674153549}]},"public_addr":"172.21.15.94:6819/3674153549","cluster_addr":"172.21.15.94:6821/3674153549","heartbeat_back_addr":"172.21.15.94:6825/3674153549","heartbeat_front_addr":"172.21.15.94:6823/3674153549","state":["exists","up"]},{"osd":5,"uuid":"ad4f4b42-77cc-431d-8996-d6f4872cb660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6803","nonce":2846740531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6805","nonce":2846740531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6809","nonce":2846740531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":2846740531},{"type":"v1","addr":"172.21.15.94:6807","nonce":2846740531}]},"public_addr":"172.21.15.94:6803/2846740531","cluster_addr":"172.21.15.94:6805/2846740531","heartbeat_back_addr":"172.21.15.94:6809/2846740531","heartbeat_front_addr":"172.21.15.94:6807/2846740531","state":["exists","up"]},{"osd":6,"uuid":"9527f394-d0d3-4abe-a5f5-91dde0438935","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6827","nonce":2189915660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6829","nonce":2189915660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6833","nonce":2189915660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2189915660},{"type":"v1","addr":"172.21.15.94:6831","nonce":2189915660}]},"public_addr":"172.21.15.94:6827/2189915660","cluster_addr":"172.21.15.94:6829/2189915660","heartbeat_back_addr":"172.21.15.94:6833/2189915660","heartbeat_front_addr":"172.21.15.94:6831/2189915660","state":["exists","up"]},{"osd":7,"uuid":"f372de29-523e-443d-92db-1012f33193c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6811","nonce":3602692462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6813","nonce":3602692462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6817","nonce":3602692462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3602692462},{"type":"v1","addr":"172.21.15.94:6815","nonce":3602692462}]},"public_addr":"172.21.15.94:6811/3602692462","cluster_addr":"172.21.15.94:6813/3602692462","heartbeat_back_addr":"172.21.15.94:6817/3602692462","heartbeat_front_addr":"172.21.15.94:6815/3602692462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.764819+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.116776+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.179676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.419344+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.405238+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.181304+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:48.439786+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T22:34:47.750273+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:6834/917223506":"2024-09-11T22:41:02.959350+0000","172.21.15.66:6832/917223506":"2024-09-11T22:41:02.959350+0000","172.21.15.66:6835/3999038135":"2024-09-11T22:38:48.643970+0000","172.21.15.66:6833/3999038135":"2024-09-11T22:38:48.643970+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1e0077f70 msgr2=0x7fe1e007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1e0077f70 0x7fe1e007a430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4151d90 tx=0x7fe1e8009040 comp rx=0 tx=0).stop 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 msgr2=0x7fe1f41509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f41509d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c410 tx=0x7fe1e400c8d0 comp rx=0 tx=0).stop 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 shutdown_connections 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe1e0077f70 0x7fe1e007a430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1f40d2950 0x7fe1f41f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1f40ce830 0x7fe1f41f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 --2- 172.21.15.66:0/2641683268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1f40cdb60 0x7fe1f41509d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 >> 172.21.15.66:0/2641683268 conn(0x7fe1f40d1e30 msgr2=0x7fe1f40b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.081+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 shutdown_connections 2024-09-10T22:41:09.083 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:09.082+0000 7fe1f9dd3640 1 -- 172.21.15.66:0/2641683268 wait complete. 2024-09-10T22:41:10.095 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T22:41:10.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:10.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.207+0000 7f731a0f7640 1 -- 172.21.15.66:0/1361667286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d0930 msgr2=0x7f73140d2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.207+0000 7f731a0f7640 1 --2- 172.21.15.66:0/1361667286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d0930 0x7f73140d2b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7304009830 tx=0x7f730402a540 comp rx=0 tx=0).stop 2024-09-10T22:41:10.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.207+0000 7f731a0f7640 1 -- 172.21.15.66:0/1361667286 shutdown_connections 2024-09-10T22:41:10.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.207+0000 7f731a0f7640 1 --2- 172.21.15.66:0/1361667286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d0930 0x7f73140d2b40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.207+0000 7f731a0f7640 1 -- 172.21.15.66:0/1361667286 >> 172.21.15.66:0/1361667286 conn(0x7f73140d3fc0 msgr2=0x7f73140d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.207+0000 7f731a0f7640 1 -- 172.21.15.66:0/1361667286 shutdown_connections 2024-09-10T22:41:10.208 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.207+0000 7f731a0f7640 1 -- 172.21.15.66:0/1361667286 wait complete. 2024-09-10T22:41:10.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.208+0000 7f731a0f7640 1 Processor -- start 2024-09-10T22:41:10.209 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.208+0000 7f731a0f7640 1 -- start start 2024-09-10T22:41:10.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f731a0f7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73140d0930 0x7f73140b9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.211 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f731a0f7640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73140d0d10 0x7f7314159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f731a0f7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d4ae0 0x7f73140cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f731a0f7640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f7314169450 con 0x7f73140d0930 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f731a0f7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f73140cfdd0 con 0x7f73140d4ae0 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f731a0f7640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f73141695d0 con 0x7f73140d0d10 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f73137fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73140d0930 0x7f73140b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f73137fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73140d0930 0x7f73140b9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36870/0 (socket says 172.21.15.66:36870) 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f73137fe640 1 -- 172.21.15.66:0/2300447207 learned_addr learned my addr 172.21.15.66:0/2300447207 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f7312ffd640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73140d0d10 0x7f7314159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f7313fff640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d4ae0 0x7f73140cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f73137fe640 1 -- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73140d0d10 msgr2=0x7f7314159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f73137fe640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73140d0d10 0x7f7314159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.209+0000 7f73137fe640 1 -- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d4ae0 msgr2=0x7f73140cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.212 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f73137fe640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d4ae0 0x7f73140cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f73137fe640 1 -- 172.21.15.66:0/2300447207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7314158ad0 con 0x7f73140d0930 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f7312ffd640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f73140d0d10 0x7f7314159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f7313fff640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73140d4ae0 0x7f73140cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f73137fe640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73140d0930 0x7f73140b9370 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f730402ab30 tx=0x7f7304030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304040070 con 0x7f73140d0930 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7304030d40 con 0x7f73140d0930 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7314158db0 con 0x7f73140d0930 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304004280 con 0x7f73140d0930 2024-09-10T22:41:10.213 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.210+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7314159370 con 0x7f73140d0930 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.211+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73140d2b40 con 0x7f73140d0930 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.212+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f730404c060 con 0x7f73140d0930 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.212+0000 7f72fbfff640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7300077f70 0x7f730007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.212+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f73040ba730 con 0x7f73140d0930 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.212+0000 7f72fbfff640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f730007ded0 0x7f73000802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.212+0000 7f7312ffd640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7300077f70 0x7f730007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.212+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 --> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73000809a0 con 0x7f730007ded0 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.212+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f73040babc0 con 0x7f73140d0930 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.213+0000 7f7313fff640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f730007ded0 0x7f73000802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.213+0000 7f7312ffd640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7300077f70 0x7f730007a430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7314154cc0 tx=0x7f7308009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.213+0000 7f7313fff640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f730007ded0 0x7f73000802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.215 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.214+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== osd.0 v2:172.21.15.66:6808/1933221850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73000809a0 con 0x7f730007ded0 2024-09-10T22:41:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.226+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 --> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7314169230 con 0x7f730007ded0 2024-09-10T22:41:10.227 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.226+0000 7f72fbfff640 1 -- 172.21.15.66:0/2300447207 <== osd.0 v2:172.21.15.66:6808/1933221850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7314169230 con 0x7f730007ded0 2024-09-10T22:41:10.227 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:10.228 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:10.228 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:10.228 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f730007ded0 msgr2=0x7f73000802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:6808/1933221850,v1:172.21.15.66:6809/1933221850] conn(0x7f730007ded0 0x7f73000802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7300077f70 msgr2=0x7f730007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7300077f70 0x7f730007a430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7314154cc0 tx=0x7f7308009040 comp rx=0 tx=0).stop 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73140d0930 msgr2=0x7f73140b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 --2- 172.21.15.66:0/2300447207 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73140d0930 0x7f73140b9370 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f730402ab30 tx=0x7f7304030650 comp rx=0 tx=0).stop 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f7313fff640 1 -- 172.21.15.66:0/2300447207 reap_dead start 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 shutdown_connections 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 >> 172.21.15.66:0/2300447207 conn(0x7f73140d3fc0 msgr2=0x7f73140c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 shutdown_connections 2024-09-10T22:41:10.229 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.227+0000 7f731a0f7640 1 -- 172.21.15.66:0/2300447207 wait complete. 2024-09-10T22:41:10.238 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T22:41:10.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 -- 172.21.15.66:0/306511888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400cfbc0 msgr2=0x7fa140167c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.345 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 --2- 172.21.15.66:0/306511888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400cfbc0 0x7fa140167c60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa138009830 tx=0x7fa13802a500 comp rx=0 tx=0).stop 2024-09-10T22:41:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 -- 172.21.15.66:0/306511888 shutdown_connections 2024-09-10T22:41:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 --2- 172.21.15.66:0/306511888 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400cfbc0 0x7fa140167c60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 --2- 172.21.15.66:0/306511888 >> v2:172.21.15.94:3300/0 conn(0x7fa1400d07a0 0x7fa1400cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 -- 172.21.15.66:0/306511888 >> 172.21.15.66:0/306511888 conn(0x7fa1400d4060 msgr2=0x7fa1400d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 -- 172.21.15.66:0/306511888 shutdown_connections 2024-09-10T22:41:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.344+0000 7fa147510640 1 -- 172.21.15.66:0/306511888 wait complete. 2024-09-10T22:41:10.346 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.345+0000 7fa147510640 1 Processor -- start 2024-09-10T22:41:10.347 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.345+0000 7fa147510640 1 -- start start 2024-09-10T22:41:10.348 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa147510640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1400cfbc0 0x7fa1400b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa147510640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400d07a0 0x7fa140158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa147510640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1400d2d70 0x7fa1400ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa147510640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa140168320 con 0x7fa1400d2d70 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa147510640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1401681a0 con 0x7fa1400cfbc0 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa147510640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa1401684a0 con 0x7fa1400d07a0 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa145aad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1400d2d70 0x7fa1400ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa145aad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1400d2d70 0x7fa1400ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36880/0 (socket says 172.21.15.66:36880) 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa145aad640 1 -- 172.21.15.66:0/2289171944 learned_addr learned my addr 172.21.15.66:0/2289171944 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa1452ac640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1400cfbc0 0x7fa1400b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.346+0000 7fa144aab640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400d07a0 0x7fa140158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa145aad640 1 -- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400d07a0 msgr2=0x7fa140158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.349 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa145aad640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400d07a0 0x7fa140158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa145aad640 1 -- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1400cfbc0 msgr2=0x7fa1400b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa145aad640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1400cfbc0 0x7fa1400b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa145aad640 1 -- 172.21.15.66:0/2289171944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1400b7da0 con 0x7fa1400d2d70 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa145aad640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1400d2d70 0x7fa1400ccbf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa138009950 tx=0x7fa138032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa131ffb640 1 -- 172.21.15.66:0/2289171944 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138040070 con 0x7fa1400d2d70 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa131ffb640 1 -- 172.21.15.66:0/2289171944 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa138005310 con 0x7fa1400d2d70 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa131ffb640 1 -- 172.21.15.66:0/2289171944 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1380047a0 con 0x7fa1400d2d70 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1400b8080 con 0x7fa1400d2d70 2024-09-10T22:41:10.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.347+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1400b8640 con 0x7fa1400d2d70 2024-09-10T22:41:10.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.348+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1400c86f0 con 0x7fa1400d2d70 2024-09-10T22:41:10.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.349+0000 7fa131ffb640 1 -- 172.21.15.66:0/2289171944 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa138004940 con 0x7fa1400d2d70 2024-09-10T22:41:10.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.349+0000 7fa131ffb640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa124077fa0 0x7fa12407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.350+0000 7fa131ffb640 1 -- 172.21.15.66:0/2289171944 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fa1380bacc0 con 0x7fa1400d2d70 2024-09-10T22:41:10.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.350+0000 7fa1452ac640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa124077fa0 0x7fa12407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.352 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.350+0000 7fa1452ac640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa124077fa0 0x7fa12407a460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa134005fd0 tx=0x7fa134009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.354+0000 7fa131ffb640 1 -- 172.21.15.66:0/2289171944 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa1400c86f0 con 0x7fa1400d2d70 2024-09-10T22:41:10.540 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.539+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa1400cf680 con 0x7fa124077fa0 2024-09-10T22:41:10.541 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.540+0000 7fa131ffb640 1 -- 172.21.15.66:0/2289171944 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa1400cf680 con 0x7fa124077fa0 2024-09-10T22:41:10.541 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.542+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa124077fa0 msgr2=0x7fa12407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.542+0000 7fa147510640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa124077fa0 0x7fa12407a460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa134005fd0 tx=0x7fa134009500 comp rx=0 tx=0).stop 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.542+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1400d2d70 msgr2=0x7fa1400ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.542+0000 7fa147510640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1400d2d70 0x7fa1400ccbf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa138009950 tx=0x7fa138032b90 comp rx=0 tx=0).stop 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 shutdown_connections 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa124077fa0 0x7fa12407a460 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa1400d2d70 0x7fa1400ccbf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa1400d07a0 0x7fa140158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 --2- 172.21.15.66:0/2289171944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1400cfbc0 0x7fa1400b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.544 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 >> 172.21.15.66:0/2289171944 conn(0x7fa1400d4060 msgr2=0x7fa1400b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 shutdown_connections 2024-09-10T22:41:10.545 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.543+0000 7fa147510640 1 -- 172.21.15.66:0/2289171944 wait complete. 2024-09-10T22:41:10.554 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T22:41:10.555 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:10.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 -- 172.21.15.66:0/1434216605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 msgr2=0x7fe9400d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 --2- 172.21.15.66:0/1434216605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 0x7fe9400d0510 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe9400af3b0 tx=0x7fe93402a560 comp rx=0 tx=0).stop 2024-09-10T22:41:10.664 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 -- 172.21.15.66:0/1434216605 shutdown_connections 2024-09-10T22:41:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 --2- 172.21.15.66:0/1434216605 >> v2:172.21.15.94:3300/0 conn(0x7fe9400cc650 0x7fe9400ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 --2- 172.21.15.66:0/1434216605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 0x7fe9400d0510 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 --2- 172.21.15.66:0/1434216605 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9401599a0 0x7fe9400b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 -- 172.21.15.66:0/1434216605 >> 172.21.15.66:0/1434216605 conn(0x7fe94006cf50 msgr2=0x7fe94006d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 -- 172.21.15.66:0/1434216605 shutdown_connections 2024-09-10T22:41:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.663+0000 7fe945394640 1 -- 172.21.15.66:0/1434216605 wait complete. 2024-09-10T22:41:10.665 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.664+0000 7fe945394640 1 Processor -- start 2024-09-10T22:41:10.666 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.664+0000 7fe945394640 1 -- start start 2024-09-10T22:41:10.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe945394640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9400cc650 0x7fe9400ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe945394640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 0x7fe9401f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe945394640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9401599a0 0x7fe9400c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe945394640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe94006ffe0 con 0x7fe9400d00d0 2024-09-10T22:41:10.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe945394640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe94006fe60 con 0x7fe9400cc650 2024-09-10T22:41:10.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe945394640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe940070160 con 0x7fe9401599a0 2024-09-10T22:41:10.668 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 0x7fe9401f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93e7fc640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 0x7fe9401f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36892/0 (socket says 172.21.15.66:36892) 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93e7fc640 1 -- 172.21.15.66:0/3794903541 learned_addr learned my addr 172.21.15.66:0/3794903541 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93effd640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9400cc650 0x7fe9400ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93f7fe640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9401599a0 0x7fe9400c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93e7fc640 1 -- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9401599a0 msgr2=0x7fe9400c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93e7fc640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe9401599a0 0x7fe9400c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.665+0000 7fe93e7fc640 1 -- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9400cc650 msgr2=0x7fe9400ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe93e7fc640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9400cc650 0x7fe9400ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe93e7fc640 1 -- 172.21.15.66:0/3794903541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9400ae900 con 0x7fe9400d00d0 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe93effd640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9400cc650 0x7fe9400ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe93e7fc640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 0x7fe9401f7f70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe934009850 tx=0x7fe9340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93403f070 con 0x7fe9400d00d0 2024-09-10T22:41:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9340047a0 con 0x7fe9400d00d0 2024-09-10T22:41:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9400ad600 con 0x7fe9400d00d0 2024-09-10T22:41:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934035030 con 0x7fe9400d00d0 2024-09-10T22:41:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.666+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9400adbc0 con 0x7fe9400d00d0 2024-09-10T22:41:10.670 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.667+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9400d0510 con 0x7fe9400d00d0 2024-09-10T22:41:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.667+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe934004bb0 con 0x7fe9400d00d0 2024-09-10T22:41:10.671 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.668+0000 7fe9277fe640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe92c077f70 0x7fe92c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.668+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fe9340baa60 con 0x7fe9400d00d0 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.668+0000 7fe93effd640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe92c077f70 0x7fe92c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.668+0000 7fe9277fe640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fe92c07ded0 0x7fe92c0802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.668+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 --> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe92c0809a0 con 0x7fe92c07ded0 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.668+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe9340bf2c0 con 0x7fe9400d00d0 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.668+0000 7fe93f7fe640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fe92c07ded0 0x7fe92c0802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.669+0000 7fe93effd640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe92c077f70 0x7fe92c07a430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe930009ea0 tx=0x7fe930005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.669+0000 7fe93f7fe640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fe92c07ded0 0x7fe92c0802f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.669+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== osd.1 v2:172.21.15.66:6800/1998688775 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe92c0809a0 con 0x7fe92c07ded0 2024-09-10T22:41:10.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.682+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 --> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe9400b3dc0 con 0x7fe92c07ded0 2024-09-10T22:41:10.683 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.682+0000 7fe9277fe640 1 -- 172.21.15.66:0/3794903541 <== osd.1 v2:172.21.15.66:6800/1998688775 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe9400b3dc0 con 0x7fe92c07ded0 2024-09-10T22:41:10.683 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:10.684 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:10.684 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:10.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.682+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fe92c07ded0 msgr2=0x7fe92c0802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.682+0000 7fe945394640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:6800/1998688775,v1:172.21.15.66:6801/1998688775] conn(0x7fe92c07ded0 0x7fe92c0802f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe92c077f70 msgr2=0x7fe92c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe92c077f70 0x7fe92c07a430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe930009ea0 tx=0x7fe930005f90 comp rx=0 tx=0).stop 2024-09-10T22:41:10.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 msgr2=0x7fe9401f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.685 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 --2- 172.21.15.66:0/3794903541 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9400d00d0 0x7fe9401f7f70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe934009850 tx=0x7fe9340038a0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe93f7fe640 1 -- 172.21.15.66:0/3794903541 reap_dead start 2024-09-10T22:41:10.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 shutdown_connections 2024-09-10T22:41:10.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 >> 172.21.15.66:0/3794903541 conn(0x7fe94006cf50 msgr2=0x7fe94015b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 shutdown_connections 2024-09-10T22:41:10.686 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.683+0000 7fe945394640 1 -- 172.21.15.66:0/3794903541 wait complete. 2024-09-10T22:41:10.694 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T22:41:10.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 -- 172.21.15.66:0/3761425827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 msgr2=0x7fd3580b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 --2- 172.21.15.66:0/3761425827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b1130 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009830 tx=0x7fd34c02a510 comp rx=0 tx=0).stop 2024-09-10T22:41:10.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 -- 172.21.15.66:0/3761425827 shutdown_connections 2024-09-10T22:41:10.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 --2- 172.21.15.66:0/3761425827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b1130 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 --2- 172.21.15.66:0/3761425827 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3580cd950 0x7fd3580688e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 -- 172.21.15.66:0/3761425827 >> 172.21.15.66:0/3761425827 conn(0x7fd358074290 msgr2=0x7fd3580746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 -- 172.21.15.66:0/3761425827 shutdown_connections 2024-09-10T22:41:10.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.799+0000 7fd35de57640 1 -- 172.21.15.66:0/3761425827 wait complete. 2024-09-10T22:41:10.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.800+0000 7fd35de57640 1 Processor -- start 2024-09-10T22:41:10.801 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.800+0000 7fd35de57640 1 -- start start 2024-09-10T22:41:10.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd35de57640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd35de57640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3580cd950 0x7fd3580b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd35de57640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3580d0ee0 0x7fd3580b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd35de57640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580b1a00 con 0x7fd3580cbe90 2024-09-10T22:41:10.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd35de57640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580b1880 con 0x7fd3580cd950 2024-09-10T22:41:10.803 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd35de57640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fd3581600d0 con 0x7fd3580d0ee0 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36910/0 (socket says 172.21.15.66:36910) 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd3577fe640 1 -- 172.21.15.66:0/1885945348 learned_addr learned my addr 172.21.15.66:0/1885945348 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd356ffd640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3580cd950 0x7fd3580b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd357fff640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3580d0ee0 0x7fd3580b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd3577fe640 1 -- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3580d0ee0 msgr2=0x7fd3580b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd3577fe640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3580d0ee0 0x7fd3580b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd3577fe640 1 -- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3580cd950 msgr2=0x7fd3580b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.801+0000 7fd3577fe640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3580cd950 0x7fd3580b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.802+0000 7fd3577fe640 1 -- 172.21.15.66:0/1885945348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3580c7580 con 0x7fd3580cbe90 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.802+0000 7fd3577fe640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b4350 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd34800ebf0 tx=0x7fd34800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.802+0000 7fd33ffff640 1 -- 172.21.15.66:0/1885945348 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348010630 con 0x7fd3580cbe90 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.802+0000 7fd33ffff640 1 -- 172.21.15.66:0/1885945348 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3480108c0 con 0x7fd3580cbe90 2024-09-10T22:41:10.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.802+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35806f550 con 0x7fd3580cbe90 2024-09-10T22:41:10.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.802+0000 7fd33ffff640 1 -- 172.21.15.66:0/1885945348 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348010d10 con 0x7fd3580cbe90 2024-09-10T22:41:10.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.802+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3580ad440 con 0x7fd3580cbe90 2024-09-10T22:41:10.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.803+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd358160600 con 0x7fd3580cbe90 2024-09-10T22:41:10.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.803+0000 7fd33ffff640 1 -- 172.21.15.66:0/1885945348 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd348018420 con 0x7fd3580cbe90 2024-09-10T22:41:10.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.804+0000 7fd33ffff640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd344077f70 0x7fd34407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:10.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.804+0000 7fd33ffff640 1 -- 172.21.15.66:0/1885945348 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fd34809a810 con 0x7fd3580cbe90 2024-09-10T22:41:10.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.804+0000 7fd356ffd640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd344077f70 0x7fd34407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:10.806 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.805+0000 7fd356ffd640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd344077f70 0x7fd34407a430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd34c034000 tx=0x7fd34c004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:10.809 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.808+0000 7fd33ffff640 1 -- 172.21.15.66:0/1885945348 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd3580c7580 con 0x7fd3580cbe90 2024-09-10T22:41:10.993 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.992+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd35815e190 con 0x7fd344077f70 2024-09-10T22:41:10.994 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.993+0000 7fd33ffff640 1 -- 172.21.15.66:0/1885945348 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd35815e190 con 0x7fd344077f70 2024-09-10T22:41:10.994 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T22:41:10.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd344077f70 msgr2=0x7fd34407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd344077f70 0x7fd34407a430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd34c034000 tx=0x7fd34c004730 comp rx=0 tx=0).stop 2024-09-10T22:41:10.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 msgr2=0x7fd3580b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:10.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b4350 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd34800ebf0 tx=0x7fd34800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 shutdown_connections 2024-09-10T22:41:10.996 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fd344077f70 0x7fd34407a430 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fd3580d0ee0 0x7fd3580b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3580cd950 0x7fd3580b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 --2- 172.21.15.66:0/1885945348 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd3580cbe90 0x7fd3580b4350 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:10.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 >> 172.21.15.66:0/1885945348 conn(0x7fd358074290 msgr2=0x7fd35815d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:10.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 shutdown_connections 2024-09-10T22:41:10.997 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:10.995+0000 7fd35de57640 1 -- 172.21.15.66:0/1885945348 wait complete. 2024-09-10T22:41:11.006 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T22:41:11.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:11.116 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2514168918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1380afa70 msgr2=0x7fe1380d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 --2- 172.21.15.66:0/2514168918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1380afa70 0x7fe1380d0740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe120009830 tx=0x7fe12002a480 comp rx=0 tx=0).stop 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2514168918 shutdown_connections 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 --2- 172.21.15.66:0/2514168918 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1380afa70 0x7fe1380d0740 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 --2- 172.21.15.66:0/2514168918 >> v2:172.21.15.94:3300/0 conn(0x7fe138151d80 0x7fe13815b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2514168918 >> 172.21.15.66:0/2514168918 conn(0x7fe13806cd10 msgr2=0x7fe13806d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2514168918 shutdown_connections 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.115+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2514168918 wait complete. 2024-09-10T22:41:11.117 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.116+0000 7fe13d9e8640 1 Processor -- start 2024-09-10T22:41:11.118 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe13d9e8640 1 -- start start 2024-09-10T22:41:11.119 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe13d9e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1380afa70 0x7fe138150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe13d9e8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1380d0c80 0x7fe1380b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe13d9e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe138151d80 0x7fe1381f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe13d9e8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe138073db0 con 0x7fe1380afa70 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe13d9e8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe138073c30 con 0x7fe138151d80 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe13d9e8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe138073f30 con 0x7fe1380d0c80 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.117+0000 7fe136ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1380afa70 0x7fe138150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1380afa70 0x7fe138150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36924/0 (socket says 172.21.15.66:36924) 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 -- 172.21.15.66:0/2204326531 learned_addr learned my addr 172.21.15.66:0/2204326531 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe1367fc640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1380d0c80 0x7fe1380b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe1377fe640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe138151d80 0x7fe1381f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 -- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1380d0c80 msgr2=0x7fe1380b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe1380d0c80 0x7fe1380b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.120 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 -- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe138151d80 msgr2=0x7fe1381f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe138151d80 0x7fe1381f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 -- 172.21.15.66:0/2204326531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe138150750 con 0x7fe1380afa70 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe136ffd640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1380afa70 0x7fe138150210 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe13000eb30 tx=0x7fe13000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130010430 con 0x7fe1380afa70 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1300106c0 con 0x7fe1380afa70 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130010b10 con 0x7fe1380afa70 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1380b0360 con 0x7fe1380afa70 2024-09-10T22:41:11.121 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.118+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1381f5800 con 0x7fe1380afa70 2024-09-10T22:41:11.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.119+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe1380d0740 con 0x7fe1380afa70 2024-09-10T22:41:11.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.120+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe130010cb0 con 0x7fe1380afa70 2024-09-10T22:41:11.122 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.120+0000 7fe12ffff640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe118077f70 0x7fe11807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.120+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fe13009a850 con 0x7fe1380afa70 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.120+0000 7fe1367fc640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe118077f70 0x7fe11807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.121+0000 7fe12ffff640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7fe11807ded0 0x7fe1180802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.121+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 --> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1180809a0 con 0x7fe11807ded0 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.121+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe13009f2c0 con 0x7fe1380afa70 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.121+0000 7fe1377fe640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7fe11807ded0 0x7fe1180802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.121+0000 7fe1367fc640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe118077f70 0x7fe11807a430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe120034000 tx=0x7fe120004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.121+0000 7fe1377fe640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7fe11807ded0 0x7fe1180802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.123 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.122+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== osd.2 v2:172.21.15.66:6816/3996225872 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1180809a0 con 0x7fe11807ded0 2024-09-10T22:41:11.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.134+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 --> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe1381f59d0 con 0x7fe11807ded0 2024-09-10T22:41:11.135 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.134+0000 7fe12ffff640 1 -- 172.21.15.66:0/2204326531 <== osd.2 v2:172.21.15.66:6816/3996225872 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe1381f59d0 con 0x7fe11807ded0 2024-09-10T22:41:11.136 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:11.136 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:11.136 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7fe11807ded0 msgr2=0x7fe1180802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:6816/3996225872,v1:172.21.15.66:6817/3996225872] conn(0x7fe11807ded0 0x7fe1180802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe118077f70 msgr2=0x7fe11807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe118077f70 0x7fe11807a430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe120034000 tx=0x7fe120004730 comp rx=0 tx=0).stop 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1380afa70 msgr2=0x7fe138150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 --2- 172.21.15.66:0/2204326531 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe1380afa70 0x7fe138150210 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe13000eb30 tx=0x7fe13000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe1377fe640 1 -- 172.21.15.66:0/2204326531 reap_dead start 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 shutdown_connections 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 >> 172.21.15.66:0/2204326531 conn(0x7fe13806cd10 msgr2=0x7fe1380ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 shutdown_connections 2024-09-10T22:41:11.137 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.135+0000 7fe13d9e8640 1 -- 172.21.15.66:0/2204326531 wait complete. 2024-09-10T22:41:11.146 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T22:41:11.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 -- 172.21.15.66:0/2365312078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604015b430 msgr2=0x7f60400ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 --2- 172.21.15.66:0/2365312078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604015b430 0x7f60400ca4b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6040070ca0 tx=0x7f603002a4a0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 -- 172.21.15.66:0/2365312078 shutdown_connections 2024-09-10T22:41:11.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 --2- 172.21.15.66:0/2365312078 >> v2:172.21.15.66:3300/0 conn(0x7f604006f810 0x7f6040164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 --2- 172.21.15.66:0/2365312078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604015b430 0x7f60400ca4b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.251 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 -- 172.21.15.66:0/2365312078 >> 172.21.15.66:0/2365312078 conn(0x7f60400b1950 msgr2=0x7f60400b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 -- 172.21.15.66:0/2365312078 shutdown_connections 2024-09-10T22:41:11.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.250+0000 7f6044d75640 1 -- 172.21.15.66:0/2365312078 wait complete. 2024-09-10T22:41:11.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.251+0000 7f6044d75640 1 Processor -- start 2024-09-10T22:41:11.252 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.251+0000 7f6044d75640 1 -- start start 2024-09-10T22:41:11.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6044d75640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f604006f810 0x7f604014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6044d75640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60400ca9f0 0x7f60401657e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6044d75640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604015b430 0x7f6040165d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6044d75640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f604006e870 con 0x7f604015b430 2024-09-10T22:41:11.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6044d75640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6040070490 con 0x7f60400ca9f0 2024-09-10T22:41:11.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6044d75640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f604006e9f0 con 0x7f604006f810 2024-09-10T22:41:11.254 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604015b430 0x7f6040165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6037fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60400ca9f0 0x7f60401657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604015b430 0x7f6040165d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36934/0 (socket says 172.21.15.66:36934) 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 -- 172.21.15.66:0/2695868581 learned_addr learned my addr 172.21.15.66:0/2695868581 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603e59c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f604006f810 0x7f604014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 -- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f604006f810 msgr2=0x7f604014db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f604006f810 0x7f604014db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 -- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60400ca9f0 msgr2=0x7f60401657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60400ca9f0 0x7f60401657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 -- 172.21.15.66:0/2695868581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60400ae620 con 0x7f604015b430 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f6037fff640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60400ca9f0 0x7f60401657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.252+0000 7f603ed9d640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604015b430 0x7f6040165d20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f603800ea60 tx=0x7f603800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.253+0000 7f6036ffd640 1 -- 172.21.15.66:0/2695868581 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800ce40 con 0x7f604015b430 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.253+0000 7f6036ffd640 1 -- 172.21.15.66:0/2695868581 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6038004b20 con 0x7f604015b430 2024-09-10T22:41:11.255 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.253+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60400ae900 con 0x7f604015b430 2024-09-10T22:41:11.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.253+0000 7f6036ffd640 1 -- 172.21.15.66:0/2695868581 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038005020 con 0x7f604015b430 2024-09-10T22:41:11.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.253+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60400aeec0 con 0x7f604015b430 2024-09-10T22:41:11.256 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.254+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6040165000 con 0x7f604015b430 2024-09-10T22:41:11.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.254+0000 7f6036ffd640 1 -- 172.21.15.66:0/2695868581 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f60380051c0 con 0x7f604015b430 2024-09-10T22:41:11.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.255+0000 7f6036ffd640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6020077f70 0x7f602007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.255+0000 7f6036ffd640 1 -- 172.21.15.66:0/2695868581 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f603809a5c0 con 0x7f604015b430 2024-09-10T22:41:11.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.255+0000 7f603e59c640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6020077f70 0x7f602007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.257 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.256+0000 7f603e59c640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6020077f70 0x7f602007a430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6030034000 tx=0x7f6030004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.260 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.259+0000 7f6036ffd640 1 -- 172.21.15.66:0/2695868581 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6040165000 con 0x7f604015b430 2024-09-10T22:41:11.444 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.443+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f60400b27b0 con 0x7f6020077f70 2024-09-10T22:41:11.445 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.444+0000 7f6036ffd640 1 -- 172.21.15.66:0/2695868581 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f60400b27b0 con 0x7f6020077f70 2024-09-10T22:41:11.445 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T22:41:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6020077f70 msgr2=0x7f602007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6020077f70 0x7f602007a430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6030034000 tx=0x7f6030004730 comp rx=0 tx=0).stop 2024-09-10T22:41:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604015b430 msgr2=0x7f6040165d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604015b430 0x7f6040165d20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f603800ea60 tx=0x7f603800ef20 comp rx=0 tx=0).stop 2024-09-10T22:41:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 shutdown_connections 2024-09-10T22:41:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6020077f70 0x7f602007a430 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f604015b430 0x7f6040165d20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60400ca9f0 0x7f60401657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 --2- 172.21.15.66:0/2695868581 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f604006f810 0x7f604014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 >> 172.21.15.66:0/2695868581 conn(0x7f60400b1950 msgr2=0x7f60400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 shutdown_connections 2024-09-10T22:41:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.446+0000 7f6044d75640 1 -- 172.21.15.66:0/2695868581 wait complete. 2024-09-10T22:41:11.457 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T22:41:11.457 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:11.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.565+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/3026554135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0073e80 msgr2=0x7fa3f00cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.566 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.565+0000 7fa3f6ad6640 1 --2- 172.21.15.66:0/3026554135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0073e80 0x7fa3f00cd210 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009830 tx=0x7fa3e402a480 comp rx=0 tx=0).stop 2024-09-10T22:41:11.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.565+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/3026554135 shutdown_connections 2024-09-10T22:41:11.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.565+0000 7fa3f6ad6640 1 --2- 172.21.15.66:0/3026554135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f0073e80 0x7fa3f00cd210 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.565+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/3026554135 >> 172.21.15.66:0/3026554135 conn(0x7fa3f00d0d70 msgr2=0x7fa3f00d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.565+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/3026554135 shutdown_connections 2024-09-10T22:41:11.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.565+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/3026554135 wait complete. 2024-09-10T22:41:11.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.566+0000 7fa3f6ad6640 1 Processor -- start 2024-09-10T22:41:11.567 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.566+0000 7fa3f6ad6640 1 -- start start 2024-09-10T22:41:11.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f6ad6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3f0073e80 0x7fa3f0151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f6ad6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f00d02b0 0x7fa3f00ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f6ad6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3f015b310 0x7fa3f006cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f6ad6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0158560 con 0x7fa3f00d02b0 2024-09-10T22:41:11.569 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f6ad6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01583e0 con 0x7fa3f0073e80 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f6ad6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f01586e0 con 0x7fa3f015b310 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3effff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f00d02b0 0x7fa3f00ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3effff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f00d02b0 0x7fa3f00ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36950/0 (socket says 172.21.15.66:36950) 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3effff640 1 -- 172.21.15.66:0/2986760839 learned_addr learned my addr 172.21.15.66:0/2986760839 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f5073640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3f015b310 0x7fa3f006cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.567+0000 7fa3f4872640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3f0073e80 0x7fa3f0151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3effff640 1 -- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3f015b310 msgr2=0x7fa3f006cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3effff640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fa3f015b310 0x7fa3f006cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3effff640 1 -- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3f0073e80 msgr2=0x7fa3f0151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3effff640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3f0073e80 0x7fa3f0151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3effff640 1 -- 172.21.15.66:0/2986760839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f0152210 con 0x7fa3f00d02b0 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3effff640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f00d02b0 0x7fa3f00ae020 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009950 tx=0x7fa3e4032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4040070 con 0x7fa3f00d02b0 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e4005310 con 0x7fa3f00d02b0 2024-09-10T22:41:11.570 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40047a0 con 0x7fa3f00d02b0 2024-09-10T22:41:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01f8820 con 0x7fa3f00d02b0 2024-09-10T22:41:11.571 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.568+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01f8de0 con 0x7fa3f00d02b0 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.569+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3f00cd210 con 0x7fa3f00d02b0 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.570+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa3e4004b50 con 0x7fa3f00d02b0 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.570+0000 7fa3ed7fa640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa3dc077f70 0x7fa3dc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.570+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fa3e40bb0c0 con 0x7fa3f00d02b0 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.570+0000 7fa3f4872640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa3dc077f70 0x7fa3dc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.570+0000 7fa3ed7fa640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7fa3dc07ded0 0x7fa3dc0802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.570+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 --> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3e4004e70 con 0x7fa3dc07ded0 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.570+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fa3e40c2050 con 0x7fa3f00d02b0 2024-09-10T22:41:11.572 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.571+0000 7fa3f5073640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7fa3dc07ded0 0x7fa3dc0802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.571+0000 7fa3f4872640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa3dc077f70 0x7fa3dc07a430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0004f60 tx=0x7fa3e0007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.571+0000 7fa3f5073640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7fa3dc07ded0 0x7fa3dc0802f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.573 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.572+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== osd.3 v2:172.21.15.66:6824/2075348943 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3e4004e70 con 0x7fa3dc07ded0 2024-09-10T22:41:11.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.584+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 --> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa3f01f9500 con 0x7fa3dc07ded0 2024-09-10T22:41:11.585 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.584+0000 7fa3ed7fa640 1 -- 172.21.15.66:0/2986760839 <== osd.3 v2:172.21.15.66:6824/2075348943 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa3f01f9500 con 0x7fa3dc07ded0 2024-09-10T22:41:11.585 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:11.586 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:11.586 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7fa3dc07ded0 msgr2=0x7fa3dc0802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.586 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:6824/2075348943,v1:172.21.15.66:6825/2075348943] conn(0x7fa3dc07ded0 0x7fa3dc0802f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa3dc077f70 msgr2=0x7fa3dc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fa3dc077f70 0x7fa3dc07a430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0004f60 tx=0x7fa3e0007e30 comp rx=0 tx=0).stop 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f00d02b0 msgr2=0x7fa3f00ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 --2- 172.21.15.66:0/2986760839 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3f00d02b0 0x7fa3f00ae020 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009950 tx=0x7fa3e4032c10 comp rx=0 tx=0).stop 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f5073640 1 -- 172.21.15.66:0/2986760839 reap_dead start 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 shutdown_connections 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 >> 172.21.15.66:0/2986760839 conn(0x7fa3f00d0d70 msgr2=0x7fa3f00b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 shutdown_connections 2024-09-10T22:41:11.587 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.585+0000 7fa3f6ad6640 1 -- 172.21.15.66:0/2986760839 wait complete. 2024-09-10T22:41:11.596 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T22:41:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.701+0000 7f02e9b94640 1 -- 172.21.15.66:0/34846917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d2d70 msgr2=0x7f02e40d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.701+0000 7f02e9b94640 1 --2- 172.21.15.66:0/34846917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d2d70 0x7f02e40d0b80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f02e40d2a60 tx=0x7f02cc02a5c0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.701+0000 7f02e9b94640 1 -- 172.21.15.66:0/34846917 shutdown_connections 2024-09-10T22:41:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.701+0000 7f02e9b94640 1 --2- 172.21.15.66:0/34846917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d2d70 0x7f02e40d0b80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.701+0000 7f02e9b94640 1 -- 172.21.15.66:0/34846917 >> 172.21.15.66:0/34846917 conn(0x7f02e40d4060 msgr2=0x7f02e40d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.701+0000 7f02e9b94640 1 -- 172.21.15.66:0/34846917 shutdown_connections 2024-09-10T22:41:11.703 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.701+0000 7f02e9b94640 1 -- 172.21.15.66:0/34846917 wait complete. 2024-09-10T22:41:11.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.702+0000 7f02e9b94640 1 Processor -- start 2024-09-10T22:41:11.704 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.703+0000 7f02e9b94640 1 -- start start 2024-09-10T22:41:11.705 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.703+0000 7f02e9b94640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d07a0 0x7f02e41ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.703+0000 7f02e9b94640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02e40d10c0 0x7f02e41eec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.703+0000 7f02e9b94640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02e40d2d70 0x7f02e41f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.703+0000 7f02e9b94640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4169500 con 0x7f02e40d07a0 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.703+0000 7f02e9b94640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f02e4169380 con 0x7f02e40d2d70 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.703+0000 7f02e9b94640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f02e4169680 con 0x7f02e40d10c0 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d07a0 0x7f02e41ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d07a0 0x7f02e41ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36976/0 (socket says 172.21.15.66:36976) 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 -- 172.21.15.66:0/411636665 learned_addr learned my addr 172.21.15.66:0/411636665 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e2ffd640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02e40d10c0 0x7f02e41eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e3fff640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02e40d2d70 0x7f02e41f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 -- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02e40d10c0 msgr2=0x7f02e41eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.706 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02e40d10c0 0x7f02e41eec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 -- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02e40d2d70 msgr2=0x7f02e41f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02e40d2d70 0x7f02e41f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 -- 172.21.15.66:0/411636665 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e40cc8a0 con 0x7f02e40d07a0 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e37fe640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d07a0 0x7f02e41ee6d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f02d400cbe0 tx=0x7f02d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02dbfff640 1 -- 172.21.15.66:0/411636665 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4016020 con 0x7f02e40d07a0 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02dbfff640 1 -- 172.21.15.66:0/411636665 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02d400ae50 con 0x7f02e40d07a0 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e40ccb80 con 0x7f02e40d07a0 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02dbfff640 1 -- 172.21.15.66:0/411636665 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4012680 con 0x7f02e40d07a0 2024-09-10T22:41:11.707 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.704+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e40cd140 con 0x7f02e40d07a0 2024-09-10T22:41:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.705+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e40c77a0 con 0x7f02e40d07a0 2024-09-10T22:41:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.706+0000 7f02dbfff640 1 -- 172.21.15.66:0/411636665 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f02d4012820 con 0x7f02e40d07a0 2024-09-10T22:41:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.706+0000 7f02dbfff640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f02c4077fa0 0x7f02c407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.707+0000 7f02dbfff640 1 -- 172.21.15.66:0/411636665 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f02d409a380 con 0x7f02e40d07a0 2024-09-10T22:41:11.708 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.707+0000 7f02e2ffd640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f02c4077fa0 0x7f02c407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:11.709 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.707+0000 7f02e2ffd640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f02c4077fa0 0x7f02c407a460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f02cc034000 tx=0x7f02cc004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:11.711 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.711+0000 7f02dbfff640 1 -- 172.21.15.66:0/411636665 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f02e40c77a0 con 0x7f02e40d07a0 2024-09-10T22:41:11.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.896+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f02e40d3e40 con 0x7f02c4077fa0 2024-09-10T22:41:11.897 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.896+0000 7f02dbfff640 1 -- 172.21.15.66:0/411636665 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f02e40d3e40 con 0x7f02c4077fa0 2024-09-10T22:41:11.898 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f02c4077fa0 msgr2=0x7f02c407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f02c4077fa0 0x7f02c407a460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f02cc034000 tx=0x7f02cc004730 comp rx=0 tx=0).stop 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d07a0 msgr2=0x7f02e41ee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d07a0 0x7f02e41ee6d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f02d400cbe0 tx=0x7f02d400a430 comp rx=0 tx=0).stop 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 shutdown_connections 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f02c4077fa0 0x7f02c407a460 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02e40d2d70 0x7f02e41f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.900 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f02e40d10c0 0x7f02e41eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 --2- 172.21.15.66:0/411636665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02e40d07a0 0x7f02e41ee6d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:11.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 >> 172.21.15.66:0/411636665 conn(0x7f02e40d4060 msgr2=0x7f02e406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:11.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 shutdown_connections 2024-09-10T22:41:11.901 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:11.899+0000 7f02e9b94640 1 -- 172.21.15.66:0/411636665 wait complete. 2024-09-10T22:41:11.910 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T22:41:11.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:12.020 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 -- 172.21.15.66:0/500700609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800d06c0 msgr2=0x7fe78015ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 --2- 172.21.15.66:0/500700609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800d06c0 0x7fe78015ba90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe7800cd740 tx=0x7fe77802aa30 comp rx=0 tx=0).stop 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 -- 172.21.15.66:0/500700609 shutdown_connections 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 --2- 172.21.15.66:0/500700609 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800d06c0 0x7fe78015ba90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 --2- 172.21.15.66:0/500700609 >> v2:172.21.15.94:3300/0 conn(0x7fe7800cf420 0x7fe78015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 --2- 172.21.15.66:0/500700609 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7800d02e0 0x7fe7800ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 -- 172.21.15.66:0/500700609 >> 172.21.15.66:0/500700609 conn(0x7fe78006cf50 msgr2=0x7fe78006d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 -- 172.21.15.66:0/500700609 shutdown_connections 2024-09-10T22:41:12.021 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.019+0000 7fe785ea9640 1 -- 172.21.15.66:0/500700609 wait complete. 2024-09-10T22:41:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.020+0000 7fe785ea9640 1 Processor -- start 2024-09-10T22:41:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.020+0000 7fe785ea9640 1 -- start start 2024-09-10T22:41:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe785ea9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cf420 0x7fe7800b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe785ea9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7800d02e0 0x7fe78014ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe785ea9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7800d06c0 0x7fe7801f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.023 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe785ea9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fe780072aa0 con 0x7fe7800cf420 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe785ea9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe780072920 con 0x7fe7800d06c0 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe785ea9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fe780072c20 con 0x7fe7800d02e0 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cf420 0x7fe7800b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cf420 0x7fe7800b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36992/0 (socket says 172.21.15.66:36992) 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 -- 172.21.15.66:0/448110571 learned_addr learned my addr 172.21.15.66:0/448110571 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77effd640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7800d02e0 0x7fe78014ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77ffff640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7800d06c0 0x7fe7801f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 -- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7800d02e0 msgr2=0x7fe78014ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fe7800d02e0 0x7fe78014ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 -- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7800d06c0 msgr2=0x7fe7801f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7800d06c0 0x7fe7801f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77f7fe640 1 -- 172.21.15.66:0/448110571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78014b260 con 0x7fe7800cf420 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.021+0000 7fe77ffff640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7800d06c0 0x7fe7801f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.022+0000 7fe77f7fe640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cf420 0x7fe7800b1fc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe77000e9f0 tx=0x7fe77000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.022+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77000cd80 con 0x7fe7800cf420 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.022+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe770005020 con 0x7fe7800cf420 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.022+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7800b0f20 con 0x7fe7800cf420 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.022+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770005450 con 0x7fe7800cf420 2024-09-10T22:41:12.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.022+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7800b14e0 con 0x7fe7800cf420 2024-09-10T22:41:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.023+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe7800ceee0 con 0x7fe7800cf420 2024-09-10T22:41:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.023+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe7700047a0 con 0x7fe7800cf420 2024-09-10T22:41:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.024+0000 7fe767fff640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe76c077f70 0x7fe76c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.024+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fe77009a580 con 0x7fe7800cf420 2024-09-10T22:41:12.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.024+0000 7fe77effd640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe76c077f70 0x7fe76c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.024+0000 7fe767fff640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7fe76c07ded0 0x7fe76c0802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.024+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 --> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe770019e70 con 0x7fe76c07ded0 2024-09-10T22:41:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.024+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe7700a1050 con 0x7fe7800cf420 2024-09-10T22:41:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.024+0000 7fe77ffff640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7fe76c07ded0 0x7fe76c0802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.025+0000 7fe77effd640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe76c077f70 0x7fe76c07a430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe7800734d0 tx=0x7fe774005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.025+0000 7fe77ffff640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7fe76c07ded0 0x7fe76c0802f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.025+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== osd.4 v2:172.21.15.94:6818/3674153549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe770019e70 con 0x7fe76c07ded0 2024-09-10T22:41:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.038+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 --> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe7800b1bf0 con 0x7fe76c07ded0 2024-09-10T22:41:12.039 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.038+0000 7fe767fff640 1 -- 172.21.15.66:0/448110571 <== osd.4 v2:172.21.15.94:6818/3674153549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe7800b1bf0 con 0x7fe76c07ded0 2024-09-10T22:41:12.040 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:12.040 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:12.040 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7fe76c07ded0 msgr2=0x7fe76c0802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6818/3674153549,v1:172.21.15.94:6819/3674153549] conn(0x7fe76c07ded0 0x7fe76c0802f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe76c077f70 msgr2=0x7fe76c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fe76c077f70 0x7fe76c07a430 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe7800734d0 tx=0x7fe774005f90 comp rx=0 tx=0).stop 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cf420 msgr2=0x7fe7800b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 --2- 172.21.15.66:0/448110571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7800cf420 0x7fe7800b1fc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe77000e9f0 tx=0x7fe77000edf0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe77ffff640 1 -- 172.21.15.66:0/448110571 reap_dead start 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 shutdown_connections 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 >> 172.21.15.66:0/448110571 conn(0x7fe78006cf50 msgr2=0x7fe780070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.041 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 shutdown_connections 2024-09-10T22:41:12.042 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.039+0000 7fe785ea9640 1 -- 172.21.15.66:0/448110571 wait complete. 2024-09-10T22:41:12.050 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.154+0000 7ffa3b80c640 1 -- 172.21.15.66:0/878545270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34070920 msgr2=0x7ffa340caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.154+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/878545270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34070920 0x7ffa340caf30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c005fd0 tx=0x7ffa2c02a5d0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.154+0000 7ffa3b80c640 1 -- 172.21.15.66:0/878545270 shutdown_connections 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.154+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/878545270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa34070920 0x7ffa340caf30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.154+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/878545270 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa341550b0 0x7ffa3415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.154+0000 7ffa3b80c640 1 -- 172.21.15.66:0/878545270 >> 172.21.15.66:0/878545270 conn(0x7ffa340b1950 msgr2=0x7ffa340b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.155+0000 7ffa3b80c640 1 -- 172.21.15.66:0/878545270 shutdown_connections 2024-09-10T22:41:12.156 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.155+0000 7ffa3b80c640 1 -- 172.21.15.66:0/878545270 wait complete. 2024-09-10T22:41:12.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.156+0000 7ffa3b80c640 1 Processor -- start 2024-09-10T22:41:12.157 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.156+0000 7ffa3b80c640 1 -- start start 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.156+0000 7ffa3b80c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa34070920 0x7ffa341537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.156+0000 7ffa3b80c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340ca720 0x7ffa340aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa3b80c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa341550b0 0x7ffa340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa3b80c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7ffa340cb470 con 0x7ffa340ca720 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa3b80c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa340cb5f0 con 0x7ffa341550b0 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa3b80c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7ffa340cb770 con 0x7ffa34070920 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340ca720 0x7ffa340aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340ca720 0x7ffa340aa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37008/0 (socket says 172.21.15.66:37008) 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 -- 172.21.15.66:0/2245553088 learned_addr learned my addr 172.21.15.66:0/2245553088 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:12.159 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa395a8640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa34070920 0x7ffa341537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa39da9640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa341550b0 0x7ffa340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 -- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa34070920 msgr2=0x7ffa341537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa34070920 0x7ffa341537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 -- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa341550b0 msgr2=0x7ffa340b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa341550b0 0x7ffa340b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 -- 172.21.15.66:0/2245553088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa340aae20 con 0x7ffa340ca720 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa38da7640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340ca720 0x7ffa340aa6f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800c450 tx=0x7ffa2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa31ffb640 1 -- 172.21.15.66:0/2245553088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28016020 con 0x7ffa340ca720 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa31ffb640 1 -- 172.21.15.66:0/2245553088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa28005400 con 0x7ffa340ca720 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa340ab100 con 0x7ffa340ca720 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.157+0000 7ffa31ffb640 1 -- 172.21.15.66:0/2245553088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa28007280 con 0x7ffa340ca720 2024-09-10T22:41:12.160 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.158+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa34159fc0 con 0x7ffa340ca720 2024-09-10T22:41:12.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.159+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3406ee90 con 0x7ffa340ca720 2024-09-10T22:41:12.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.159+0000 7ffa31ffb640 1 -- 172.21.15.66:0/2245553088 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffa28007420 con 0x7ffa340ca720 2024-09-10T22:41:12.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.160+0000 7ffa31ffb640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa20077f70 0x7ffa2007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.161 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.160+0000 7ffa31ffb640 1 -- 172.21.15.66:0/2245553088 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7ffa280992e0 con 0x7ffa340ca720 2024-09-10T22:41:12.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.160+0000 7ffa395a8640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa20077f70 0x7ffa2007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.162 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.160+0000 7ffa395a8640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa20077f70 0x7ffa2007a430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffa24005fd0 tx=0x7ffa24009500 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.164+0000 7ffa31ffb640 1 -- 172.21.15.66:0/2245553088 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ffa28020020 con 0x7ffa340ca720 2024-09-10T22:41:12.350 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.349+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7ffa3406f430 con 0x7ffa20077f70 2024-09-10T22:41:12.351 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.350+0000 7ffa31ffb640 1 -- 172.21.15.66:0/2245553088 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffa3406f430 con 0x7ffa20077f70 2024-09-10T22:41:12.351 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T22:41:12.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa20077f70 msgr2=0x7ffa2007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa20077f70 0x7ffa2007a430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffa24005fd0 tx=0x7ffa24009500 comp rx=0 tx=0).stop 2024-09-10T22:41:12.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340ca720 msgr2=0x7ffa340aa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340ca720 0x7ffa340aa6f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800c450 tx=0x7ffa2800c910 comp rx=0 tx=0).stop 2024-09-10T22:41:12.353 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 shutdown_connections 2024-09-10T22:41:12.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7ffa20077f70 0x7ffa2007a430 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa341550b0 0x7ffa340b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa340ca720 0x7ffa340aa6f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 --2- 172.21.15.66:0/2245553088 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7ffa34070920 0x7ffa341537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 >> 172.21.15.66:0/2245553088 conn(0x7ffa340b1950 msgr2=0x7ffa34153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 shutdown_connections 2024-09-10T22:41:12.354 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.352+0000 7ffa3b80c640 1 -- 172.21.15.66:0/2245553088 wait complete. 2024-09-10T22:41:12.363 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T22:41:12.363 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.472+0000 7fbecf3c6640 1 -- 172.21.15.66:0/3346525900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec80d0bf0 msgr2=0x7fbec80d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.472+0000 7fbecf3c6640 1 --2- 172.21.15.66:0/3346525900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec80d0bf0 0x7fbec80d1060 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbec806ec30 tx=0x7fbeb802a560 comp rx=0 tx=0).stop 2024-09-10T22:41:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.473+0000 7fbecf3c6640 1 -- 172.21.15.66:0/3346525900 shutdown_connections 2024-09-10T22:41:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.473+0000 7fbecf3c6640 1 --2- 172.21.15.66:0/3346525900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec80d0bf0 0x7fbec80d1060 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.473+0000 7fbecf3c6640 1 --2- 172.21.15.66:0/3346525900 >> v2:172.21.15.94:3300/0 conn(0x7fbec8158510 0x7fbec815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.473+0000 7fbecf3c6640 1 -- 172.21.15.66:0/3346525900 >> 172.21.15.66:0/3346525900 conn(0x7fbec806dc40 msgr2=0x7fbec806e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.474 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.473+0000 7fbecf3c6640 1 -- 172.21.15.66:0/3346525900 shutdown_connections 2024-09-10T22:41:12.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.473+0000 7fbecf3c6640 1 -- 172.21.15.66:0/3346525900 wait complete. 2024-09-10T22:41:12.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.474+0000 7fbecf3c6640 1 Processor -- start 2024-09-10T22:41:12.475 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.474+0000 7fbecf3c6640 1 -- start start 2024-09-10T22:41:12.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecf3c6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec8073840 0x7fbec81f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecf3c6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbec80d0bf0 0x7fbec80d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecf3c6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec8158510 0x7fbec8153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecf3c6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fbec815e8d0 con 0x7fbec8158510 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecf3c6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbec815e750 con 0x7fbec8073840 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecf3c6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fbec815f3a0 con 0x7fbec80d0bf0 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec8158510 0x7fbec8153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec8158510 0x7fbec8153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37026/0 (socket says 172.21.15.66:37026) 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 -- 172.21.15.66:0/4039636344 learned_addr learned my addr 172.21.15.66:0/4039636344 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecc961640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbec80d0bf0 0x7fbec80d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd162640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec8073840 0x7fbec81f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 -- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbec80d0bf0 msgr2=0x7fbec80d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbec80d0bf0 0x7fbec80d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 -- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec8073840 msgr2=0x7fbec81f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec8073840 0x7fbec81f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 -- 172.21.15.66:0/4039636344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec81f8a90 con 0x7fbec8158510 2024-09-10T22:41:12.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecc961640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fbec80d0bf0 0x7fbec80d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:12.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd162640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbec8073840 0x7fbec81f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:12.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecd963640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec8158510 0x7fbec8153910 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c770 tx=0x7fbec000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0003da0 con 0x7fbec8158510 2024-09-10T22:41:12.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbec0005070 con 0x7fbec8158510 2024-09-10T22:41:12.479 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec81f48f0 con 0x7fbec8158510 2024-09-10T22:41:12.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.475+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0005500 con 0x7fbec8158510 2024-09-10T22:41:12.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.476+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec81f5190 con 0x7fbec8158510 2024-09-10T22:41:12.480 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.477+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbec80d1060 con 0x7fbec8158510 2024-09-10T22:41:12.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.477+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbec000a5e0 con 0x7fbec8158510 2024-09-10T22:41:12.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbebdffb640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbeac077f70 0x7fbeac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fbec0099840 con 0x7fbec8158510 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbebdffb640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7fbeac07ded0 0x7fbeac0802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 --> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbeac0809a0 con 0x7fbeac07ded0 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fbec009e2c0 con 0x7fbec8158510 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbecd162640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbeac077f70 0x7fbeac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbecc961640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7fbeac07ded0 0x7fbeac0802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbecd162640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbeac077f70 0x7fbeac07a430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009ab0 tx=0x7fbec4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.478+0000 7fbecc961640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7fbeac07ded0 0x7fbeac0802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.479+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== osd.5 v2:172.21.15.94:6802/2846740531 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbeac0809a0 con 0x7fbeac07ded0 2024-09-10T22:41:12.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.492+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 --> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbec815bc10 con 0x7fbeac07ded0 2024-09-10T22:41:12.493 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.492+0000 7fbebdffb640 1 -- 172.21.15.66:0/4039636344 <== osd.5 v2:172.21.15.94:6802/2846740531 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbec815bc10 con 0x7fbeac07ded0 2024-09-10T22:41:12.493 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:12.493 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:12.493 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:12.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.492+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7fbeac07ded0 msgr2=0x7fbeac0802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.492+0000 7fbecf3c6640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6802/2846740531,v1:172.21.15.94:6803/2846740531] conn(0x7fbeac07ded0 0x7fbeac0802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.494 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbeac077f70 msgr2=0x7fbeac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fbeac077f70 0x7fbeac07a430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009ab0 tx=0x7fbec4005f90 comp rx=0 tx=0).stop 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec8158510 msgr2=0x7fbec8153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 --2- 172.21.15.66:0/4039636344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbec8158510 0x7fbec8153910 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c770 tx=0x7fbec000cc30 comp rx=0 tx=0).stop 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecd963640 1 -- 172.21.15.66:0/4039636344 reap_dead start 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 shutdown_connections 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 >> 172.21.15.66:0/4039636344 conn(0x7fbec806dc40 msgr2=0x7fbec80ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 shutdown_connections 2024-09-10T22:41:12.495 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.493+0000 7fbecf3c6640 1 -- 172.21.15.66:0/4039636344 wait complete. 2024-09-10T22:41:12.504 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T22:41:12.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.609+0000 7f6adb117640 1 -- 172.21.15.66:0/820653542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 msgr2=0x7f6ad4070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.610 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.609+0000 7f6adb117640 1 --2- 172.21.15.66:0/820653542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 0x7f6ad4070210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009860 tx=0x7f6ac002a5a0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.609+0000 7f6adb117640 1 -- 172.21.15.66:0/820653542 shutdown_connections 2024-09-10T22:41:12.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.609+0000 7f6adb117640 1 --2- 172.21.15.66:0/820653542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 0x7f6ad4070210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.609+0000 7f6adb117640 1 -- 172.21.15.66:0/820653542 >> 172.21.15.66:0/820653542 conn(0x7f6ad40b1950 msgr2=0x7f6ad40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.609+0000 7f6adb117640 1 -- 172.21.15.66:0/820653542 shutdown_connections 2024-09-10T22:41:12.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.609+0000 7f6adb117640 1 -- 172.21.15.66:0/820653542 wait complete. 2024-09-10T22:41:12.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.610+0000 7f6adb117640 1 Processor -- start 2024-09-10T22:41:12.611 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.610+0000 7f6adb117640 1 -- start start 2024-09-10T22:41:12.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6adb117640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ad4070750 0x7f6ad4153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6adb117640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad41550b0 0x7f6ad406f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.613 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6adb117640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 0x7f6ad406f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6adb117640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40cb7c0 con 0x7f6ad4070750 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6adb117640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40cb640 con 0x7f6ad41550b0 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6adb117640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad40cb940 con 0x7f6ad415b200 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6ad8eb3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ad4070750 0x7f6ad4153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.611+0000 7f6ad8eb3640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ad4070750 0x7f6ad4153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37034/0 (socket says 172.21.15.66:37034) 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad8eb3640 1 -- 172.21.15.66:0/934006619 learned_addr learned my addr 172.21.15.66:0/934006619 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad96b4640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 0x7f6ad406f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad3fff640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad41550b0 0x7f6ad406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad8eb3640 1 -- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 msgr2=0x7f6ad406f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad8eb3640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 0x7f6ad406f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad8eb3640 1 -- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad41550b0 msgr2=0x7f6ad406f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.614 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad8eb3640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad41550b0 0x7f6ad406f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad8eb3640 1 -- 172.21.15.66:0/934006619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad40c9c40 con 0x7f6ad4070750 2024-09-10T22:41:12.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad8eb3640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ad4070750 0x7f6ad4153100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800c4d0 tx=0x7f6ac800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad17fa640 1 -- 172.21.15.66:0/934006619 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8016020 con 0x7f6ad4070750 2024-09-10T22:41:12.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad17fa640 1 -- 172.21.15.66:0/934006619 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ac8012a00 con 0x7f6ad4070750 2024-09-10T22:41:12.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad40c9f20 con 0x7f6ad4070750 2024-09-10T22:41:12.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6ad17fa640 1 -- 172.21.15.66:0/934006619 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8011380 con 0x7f6ad4070750 2024-09-10T22:41:12.615 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.612+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41e61e0 con 0x7f6ad4070750 2024-09-10T22:41:12.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.613+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad406ee90 con 0x7f6ad4070750 2024-09-10T22:41:12.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.614+0000 7f6ad17fa640 1 -- 172.21.15.66:0/934006619 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6ac8011520 con 0x7f6ad4070750 2024-09-10T22:41:12.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.614+0000 7f6ad17fa640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ab8077f70 0x7f6ab807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.615+0000 7f6ad17fa640 1 -- 172.21.15.66:0/934006619 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f6ac8099990 con 0x7f6ad4070750 2024-09-10T22:41:12.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.615+0000 7f6ad3fff640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ab8077f70 0x7f6ab807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.616 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.615+0000 7f6ad3fff640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ab8077f70 0x7f6ab807a430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0034000 tx=0x7f6ac0004730 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.620 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.619+0000 7f6ad17fa640 1 -- 172.21.15.66:0/934006619 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6ac8020030 con 0x7f6ad4070750 2024-09-10T22:41:12.804 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.803+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f6ad40cf0f0 con 0x7f6ab8077f70 2024-09-10T22:41:12.805 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.804+0000 7f6ad17fa640 1 -- 172.21.15.66:0/934006619 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6ad40cf0f0 con 0x7f6ab8077f70 2024-09-10T22:41:12.805 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T22:41:12.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ab8077f70 msgr2=0x7f6ab807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ab8077f70 0x7f6ab807a430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0034000 tx=0x7f6ac0004730 comp rx=0 tx=0).stop 2024-09-10T22:41:12.807 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ad4070750 msgr2=0x7f6ad4153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ad4070750 0x7f6ad4153100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800c4d0 tx=0x7f6ac800c990 comp rx=0 tx=0).stop 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 shutdown_connections 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ab8077f70 0x7f6ab807a430 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6ad415b200 0x7f6ad406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6ad41550b0 0x7f6ad406f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 --2- 172.21.15.66:0/934006619 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6ad4070750 0x7f6ad4153100 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 >> 172.21.15.66:0/934006619 conn(0x7f6ad40b1950 msgr2=0x7f6ad415a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 shutdown_connections 2024-09-10T22:41:12.808 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.806+0000 7f6adb117640 1 -- 172.21.15.66:0/934006619 wait complete. 2024-09-10T22:41:12.818 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T22:41:12.818 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 -- 172.21.15.66:0/2573242915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0d10 msgr2=0x7fb1d80b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 --2- 172.21.15.66:0/2573242915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0d10 0x7fb1d80b3be0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009be0 tx=0x7fb1d002a8d0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 -- 172.21.15.66:0/2573242915 shutdown_connections 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 --2- 172.21.15.66:0/2573242915 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0d10 0x7fb1d80b3be0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 --2- 172.21.15.66:0/2573242915 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1d80d4ae0 0x7fb1d80d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 -- 172.21.15.66:0/2573242915 >> 172.21.15.66:0/2573242915 conn(0x7fb1d80d3fc0 msgr2=0x7fb1d80d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 -- 172.21.15.66:0/2573242915 shutdown_connections 2024-09-10T22:41:12.927 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.925+0000 7fb1ded9c640 1 -- 172.21.15.66:0/2573242915 wait complete. 2024-09-10T22:41:12.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.927+0000 7fb1ded9c640 1 Processor -- start 2024-09-10T22:41:12.928 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.927+0000 7fb1ded9c640 1 -- start start 2024-09-10T22:41:12.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.927+0000 7fb1ded9c640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0930 0x7fb1d81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.930 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.927+0000 7fb1ded9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d80d0d10 0x7fb1d80b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.927+0000 7fb1ded9c640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1d80d4ae0 0x7fb1d80cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1ded9c640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d80ceeb0 con 0x7fb1d80d0930 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1ded9c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d80ced30 con 0x7fb1d80d0d10 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1ded9c640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d80cf030 con 0x7fb1d80d4ae0 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0930 0x7fb1d81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0930 0x7fb1d81587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37064/0 (socket says 172.21.15.66:37064) 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 -- 172.21.15.66:0/732355421 learned_addr learned my addr 172.21.15.66:0/732355421 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dd339640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1d80d4ae0 0x7fb1d80cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1d7fff640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d80d0d10 0x7fb1d80b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 -- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1d80d4ae0 msgr2=0x7fb1d80cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fb1d80d4ae0 0x7fb1d80cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 -- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d80d0d10 msgr2=0x7fb1d80b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.931 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d80d0d10 0x7fb1d80b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 -- 172.21.15.66:0/732355421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d8158d00 con 0x7fb1d80d0930 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1d7fff640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1d80d0d10 0x7fb1d80b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1dcb38640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0930 0x7fb1d81587c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c500 tx=0x7fb1cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc017070 con 0x7fb1d80d0930 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1cc0035f0 con 0x7fb1d80d0930 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d81e7080 con 0x7fb1d80d0930 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc003a40 con 0x7fb1d80d0930 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.928+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d81e7640 con 0x7fb1d80d0930 2024-09-10T22:41:12.932 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.929+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1d80d2b40 con 0x7fb1d80d0930 2024-09-10T22:41:12.933 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.930+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb1cc003be0 con 0x7fb1d80d0930 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.930+0000 7fb1d57fa640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb1bc077f70 0x7fb1bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fb1cc099140 con 0x7fb1d80d0930 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1d57fa640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7fb1bc07ded0 0x7fb1bc0802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 --> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1cc008e50 con 0x7fb1bc07ded0 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc0a0050 con 0x7fb1d80d0930 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1d7fff640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb1bc077f70 0x7fb1bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1dd339640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7fb1bc07ded0 0x7fb1bc0802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1d7fff640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb1bc077f70 0x7fb1bc07a430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb1d81e9300 tx=0x7fb1c4005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.931+0000 7fb1dd339640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7fb1bc07ded0 0x7fb1bc0802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:12.934 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.932+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== osd.6 v2:172.21.15.94:6826/2189915660 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1cc008e50 con 0x7fb1bc07ded0 2024-09-10T22:41:12.945 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 --> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb1d806f140 con 0x7fb1bc07ded0 2024-09-10T22:41:12.946 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1d57fa640 1 -- 172.21.15.66:0/732355421 <== osd.6 v2:172.21.15.94:6826/2189915660 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb1d806f140 con 0x7fb1bc07ded0 2024-09-10T22:41:12.946 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:12.946 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:12.946 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:12.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7fb1bc07ded0 msgr2=0x7fb1bc0802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1ded9c640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6826/2189915660,v1:172.21.15.94:6827/2189915660] conn(0x7fb1bc07ded0 0x7fb1bc0802f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb1bc077f70 msgr2=0x7fb1bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1ded9c640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fb1bc077f70 0x7fb1bc07a430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb1d81e9300 tx=0x7fb1c4005f90 comp rx=0 tx=0).stop 2024-09-10T22:41:12.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0930 msgr2=0x7fb1d81587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:12.947 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.945+0000 7fb1ded9c640 1 --2- 172.21.15.66:0/732355421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1d80d0930 0x7fb1d81587c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c500 tx=0x7fb1cc00c9c0 comp rx=0 tx=0).stop 2024-09-10T22:41:12.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.946+0000 7fb1dd339640 1 -- 172.21.15.66:0/732355421 reap_dead start 2024-09-10T22:41:12.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.946+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 shutdown_connections 2024-09-10T22:41:12.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.946+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 >> 172.21.15.66:0/732355421 conn(0x7fb1d80d3fc0 msgr2=0x7fb1d80757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:12.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.946+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 shutdown_connections 2024-09-10T22:41:12.948 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:12.946+0000 7fb1ded9c640 1 -- 172.21.15.66:0/732355421 wait complete. 2024-09-10T22:41:12.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T22:41:13.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.063+0000 7f6cc99e8640 1 -- 172.21.15.66:0/4274669313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00052e0 msgr2=0x7f6cb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.063+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/4274669313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00052e0 0x7f6cb000b9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40013d0 tx=0x7f6cb0005830 comp rx=0 tx=0).stop 2024-09-10T22:41:13.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.063+0000 7f6cc99e8640 1 -- 172.21.15.66:0/4274669313 shutdown_connections 2024-09-10T22:41:13.064 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.063+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/4274669313 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00052e0 0x7f6cb000b9c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.063+0000 7f6cc99e8640 1 -- 172.21.15.66:0/4274669313 >> 172.21.15.66:0/4274669313 conn(0x7f6cc40b1950 msgr2=0x7f6cc40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:13.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.063+0000 7f6cc99e8640 1 -- 172.21.15.66:0/4274669313 shutdown_connections 2024-09-10T22:41:13.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.063+0000 7f6cc99e8640 1 -- 172.21.15.66:0/4274669313 wait complete. 2024-09-10T22:41:13.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.064+0000 7f6cc99e8640 1 Processor -- start 2024-09-10T22:41:13.065 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.064+0000 7f6cc99e8640 1 -- start start 2024-09-10T22:41:13.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc99e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb00052e0 0x7f6cc40ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc99e8640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6cc40aef60 0x7f6cc415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc99e8640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cc41f58b0 0x7f6cc40ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc99e8640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40ca630 con 0x7f6cc41f58b0 2024-09-10T22:41:13.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc99e8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc40ca4b0 con 0x7f6cb00052e0 2024-09-10T22:41:13.067 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc99e8640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc40ca7b0 con 0x7f6cc40aef60 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cc41f58b0 0x7f6cc40ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cc41f58b0 0x7f6cc40ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37082/0 (socket says 172.21.15.66:37082) 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 -- 172.21.15.66:0/461553922 learned_addr learned my addr 172.21.15.66:0/461553922 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc27fc640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6cc40aef60 0x7f6cc415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc2ffd640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb00052e0 0x7f6cc40ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 -- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6cc40aef60 msgr2=0x7f6cc415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6cc40aef60 0x7f6cc415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 -- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb00052e0 msgr2=0x7f6cc40ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb00052e0 0x7f6cc40ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.065+0000 7f6cc37fe640 1 -- 172.21.15.66:0/461553922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc40adec0 con 0x7f6cc41f58b0 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.066+0000 7f6cc37fe640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cc41f58b0 0x7f6cc40ce7c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400c700 tx=0x7f6cb4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.066+0000 7f6cbbfff640 1 -- 172.21.15.66:0/461553922 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4039070 con 0x7f6cc41f58b0 2024-09-10T22:41:13.068 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.066+0000 7f6cbbfff640 1 -- 172.21.15.66:0/461553922 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cb402bc60 con 0x7f6cc41f58b0 2024-09-10T22:41:13.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.066+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc40ae1a0 con 0x7f6cc41f58b0 2024-09-10T22:41:13.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.066+0000 7f6cbbfff640 1 -- 172.21.15.66:0/461553922 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4006330 con 0x7f6cc41f58b0 2024-09-10T22:41:13.069 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.066+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc40ae760 con 0x7f6cc41f58b0 2024-09-10T22:41:13.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.067+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc40701c0 con 0x7f6cc41f58b0 2024-09-10T22:41:13.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.067+0000 7f6cbbfff640 1 -- 172.21.15.66:0/461553922 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6cb4046060 con 0x7f6cc41f58b0 2024-09-10T22:41:13.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.068+0000 7f6cbbfff640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ca4077fa0 0x7f6ca407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.068+0000 7f6cbbfff640 1 -- 172.21.15.66:0/461553922 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f6cb40b90b0 con 0x7f6cc41f58b0 2024-09-10T22:41:13.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.068+0000 7f6cc2ffd640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ca4077fa0 0x7f6ca407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.070 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.068+0000 7f6cc2ffd640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ca4077fa0 0x7f6ca407a460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc006680 tx=0x7f6cbc006b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.073 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.072+0000 7f6cbbfff640 1 -- 172.21.15.66:0/461553922 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6cb4081950 con 0x7f6cc41f58b0 2024-09-10T22:41:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.257+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f6cc41f5d40 con 0x7f6ca4077fa0 2024-09-10T22:41:13.259 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.258+0000 7f6cbbfff640 1 -- 172.21.15.66:0/461553922 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6cc41f5d40 con 0x7f6ca4077fa0 2024-09-10T22:41:13.259 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T22:41:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ca4077fa0 msgr2=0x7f6ca407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ca4077fa0 0x7f6ca407a460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc006680 tx=0x7f6cbc006b70 comp rx=0 tx=0).stop 2024-09-10T22:41:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cc41f58b0 msgr2=0x7f6cc40ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cc41f58b0 0x7f6cc40ce7c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400c700 tx=0x7f6cb4003ba0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 shutdown_connections 2024-09-10T22:41:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f6ca4077fa0 0x7f6ca407a460 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cc41f58b0 0x7f6cc40ce7c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f6cc40aef60 0x7f6cc415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 --2- 172.21.15.66:0/461553922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb00052e0 0x7f6cc40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 >> 172.21.15.66:0/461553922 conn(0x7f6cc40b1950 msgr2=0x7f6cc40b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 shutdown_connections 2024-09-10T22:41:13.263 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.261+0000 7f6cc99e8640 1 -- 172.21.15.66:0/461553922 wait complete. 2024-09-10T22:41:13.273 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T22:41:13.273 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T22:41:13.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.382+0000 7f2297252640 1 -- 172.21.15.66:0/800531240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 msgr2=0x7f227800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.382+0000 7f2297252640 1 --2- 172.21.15.66:0/800531240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 0x7f227800b9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2280009b50 tx=0x7f2278005830 comp rx=0 tx=0).stop 2024-09-10T22:41:13.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.382+0000 7f2297252640 1 -- 172.21.15.66:0/800531240 shutdown_connections 2024-09-10T22:41:13.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.382+0000 7f2297252640 1 --2- 172.21.15.66:0/800531240 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 0x7f227800b9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.384 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.382+0000 7f2297252640 1 -- 172.21.15.66:0/800531240 >> 172.21.15.66:0/800531240 conn(0x7f229006cd10 msgr2=0x7f229006d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:13.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.382+0000 7f2297252640 1 -- 172.21.15.66:0/800531240 shutdown_connections 2024-09-10T22:41:13.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.382+0000 7f2297252640 1 -- 172.21.15.66:0/800531240 wait complete. 2024-09-10T22:41:13.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.383+0000 7f2297252640 1 Processor -- start 2024-09-10T22:41:13.385 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.383+0000 7f2297252640 1 -- start start 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2297252640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 0x7f22901f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2297252640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22900ae040 0x7f2290144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2297252640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22900abff0 0x7f22900ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2297252640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f229015fda0 con 0x7f2278005210 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2297252640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f229015fc20 con 0x7f22900abff0 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2297252640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f22900d0740 con 0x7f22900ae040 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 0x7f22901f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 0x7f22901f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37094/0 (socket says 172.21.15.66:37094) 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 -- 172.21.15.66:0/3804877489 learned_addr learned my addr 172.21.15.66:0/3804877489 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f22957ef640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22900abff0 0x7f22900ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f228ffff640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22900ae040 0x7f2290144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 -- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22900ae040 msgr2=0x7f2290144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.387 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22900ae040 0x7f2290144660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 -- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22900abff0 msgr2=0x7f22900ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f22900abff0 0x7f22900ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 -- 172.21.15.66:0/3804877489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22900ac9b0 con 0x7f2278005210 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f228ffff640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f22900ae040 0x7f2290144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.384+0000 7f2294fee640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 0x7f22901f6940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2288002230 tx=0x7f22880064f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.385+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288015070 con 0x7f2278005210 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.385+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f228800fa90 con 0x7f2278005210 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.385+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22900ae800 con 0x7f2278005210 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.385+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288011430 con 0x7f2278005210 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.385+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22900aedc0 con 0x7f2278005210 2024-09-10T22:41:13.388 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.386+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22900d0c40 con 0x7f2278005210 2024-09-10T22:41:13.389 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.386+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f22880115d0 con 0x7f2278005210 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f228d7fa640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f227c077f70 0x7f227c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f228809a930 con 0x7f2278005210 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f228d7fa640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f227c07ded0 0x7f227c0802f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f228ffff640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f227c077f70 0x7f227c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 --> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f228801ce50 con 0x7f227c07ded0 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f228809f2a0 con 0x7f2278005210 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f22957ef640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f227c07ded0 0x7f227c0802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.387+0000 7f228ffff640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f227c077f70 0x7f227c07a430 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2284000fd0 tx=0x7f22840014f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.388+0000 7f22957ef640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f227c07ded0 0x7f227c0802f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.390 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.388+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== osd.7 v2:172.21.15.94:6810/3602692462 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f228801ce50 con 0x7f227c07ded0 2024-09-10T22:41:13.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.401+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 --> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f22900cfd60 con 0x7f227c07ded0 2024-09-10T22:41:13.402 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.401+0000 7f228d7fa640 1 -- 172.21.15.66:0/3804877489 <== osd.7 v2:172.21.15.94:6810/3602692462 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f22900cfd60 con 0x7f227c07ded0 2024-09-10T22:41:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T22:41:13.402 INFO:teuthology.orchestra.run.smithi066.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T22:41:13.402 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.401+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f227c07ded0 msgr2=0x7f227c0802f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.401+0000 7f2297252640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6810/3602692462,v1:172.21.15.94:6811/3602692462] conn(0x7f227c07ded0 0x7f227c0802f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f227c077f70 msgr2=0x7f227c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f227c077f70 0x7f227c07a430 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2284000fd0 tx=0x7f22840014f0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 msgr2=0x7f22901f6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 --2- 172.21.15.66:0/3804877489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2278005210 0x7f22901f6940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2288002230 tx=0x7f22880064f0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f22957ef640 1 -- 172.21.15.66:0/3804877489 reap_dead start 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 shutdown_connections 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 >> 172.21.15.66:0/3804877489 conn(0x7f229006cd10 msgr2=0x7f22900b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 shutdown_connections 2024-09-10T22:41:13.404 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.402+0000 7f2297252640 1 -- 172.21.15.66:0/3804877489 wait complete. 2024-09-10T22:41:13.413 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T22:41:13.519 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 -- 172.21.15.66:0/1034936219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0cd560 msgr2=0x7f91fc0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 --2- 172.21.15.66:0/1034936219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0cd560 0x7f91fc0adda0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f91fc06f7b0 tx=0x7f91ec02a4d0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 -- 172.21.15.66:0/1034936219 shutdown_connections 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 --2- 172.21.15.66:0/1034936219 >> v2:172.21.15.94:3300/0 conn(0x7f91fc0ae2e0 0x7f91fc06d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 --2- 172.21.15.66:0/1034936219 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91fc0cc860 0x7f91fc06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 --2- 172.21.15.66:0/1034936219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0cd560 0x7f91fc0adda0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 -- 172.21.15.66:0/1034936219 >> 172.21.15.66:0/1034936219 conn(0x7f91fc0cd9e0 msgr2=0x7f91fc0cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 -- 172.21.15.66:0/1034936219 shutdown_connections 2024-09-10T22:41:13.520 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.518+0000 7f9202cc6640 1 -- 172.21.15.66:0/1034936219 wait complete. 2024-09-10T22:41:13.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.519+0000 7f9202cc6640 1 Processor -- start 2024-09-10T22:41:13.521 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.519+0000 7f9202cc6640 1 -- start start 2024-09-10T22:41:13.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9202cc6640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0ae2e0 0x7f91fc0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9202cc6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91fc0cc860 0x7f91fc14f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.522 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9202cc6640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91fc0cd560 0x7f91fc158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9202cc6640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc0cb940 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9202cc6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc0cb7c0 con 0x7f91fc0cc860 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9202cc6640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f91fc0cbac0 con 0x7f91fc0cd560 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9200a62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0ae2e0 0x7f91fc0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9200a62640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0ae2e0 0x7f91fc0b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53064/0 (socket says 172.21.15.66:53064) 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.520+0000 7f9200a62640 1 -- 172.21.15.66:0/3549256279 learned_addr learned my addr 172.21.15.66:0/3549256279 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f91fbfff640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91fc0cc860 0x7f91fc14f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9201263640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91fc0cd560 0x7f91fc158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9200a62640 1 -- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91fc0cd560 msgr2=0x7f91fc158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9200a62640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91fc0cd560 0x7f91fc158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9200a62640 1 -- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91fc0cc860 msgr2=0x7f91fc14f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9200a62640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91fc0cc860 0x7f91fc14f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.523 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9200a62640 1 -- 172.21.15.66:0/3549256279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc0ca3f0 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9201263640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91fc0cd560 0x7f91fc158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:41:13.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9200a62640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0ae2e0 0x7f91fc0b47c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f91ec034040 tx=0x7f91ec0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f91f97fa640 1 -- 172.21.15.66:0/3549256279 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec03e030 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f91f97fa640 1 -- 172.21.15.66:0/3549256279 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91ec0047a0 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f91f97fa640 1 -- 172.21.15.66:0/3549256279 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec035040 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc0c9630 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.524 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.521+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc0c9bf0 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.522+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91fc0adda0 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.523+0000 7f91f97fa640 1 -- 172.21.15.66:0/3549256279 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f91ec004bf0 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.523+0000 7f91f97fa640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f91e8077f70 0x7f91e807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.523+0000 7f91f97fa640 1 -- 172.21.15.66:0/3549256279 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f91ec0ba470 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.523+0000 7f91fbfff640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f91e8077f70 0x7f91e807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.525 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.524+0000 7f91fbfff640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f91e8077f70 0x7f91e807a430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0b0e30 tx=0x7f91f0005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.528 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.528+0000 7f91f97fa640 1 -- 172.21.15.66:0/3549256279 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f91ec082c50 con 0x7f91fc0ae2e0 2024-09-10T22:41:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.716+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f91fc0b6060 con 0x7f91e8077f70 2024-09-10T22:41:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.716+0000 7f91f97fa640 1 -- 172.21.15.66:0/3549256279 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f91f000fb90 con 0x7f91e8077f70 2024-09-10T22:41:13.717 INFO:teuthology.orchestra.run.smithi066.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T22:41:13.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f91e8077f70 msgr2=0x7f91e807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.719 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f91e8077f70 0x7f91e807a430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0b0e30 tx=0x7f91f0005f90 comp rx=0 tx=0).stop 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0ae2e0 msgr2=0x7f91fc0b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0ae2e0 0x7f91fc0b47c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f91ec034040 tx=0x7f91ec0039e0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 shutdown_connections 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f91e8077f70 0x7f91e807a430 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f91fc0cd560 0x7f91fc158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91fc0cc860 0x7f91fc14f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 --2- 172.21.15.66:0/3549256279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91fc0ae2e0 0x7f91fc0b47c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 >> 172.21.15.66:0/3549256279 conn(0x7f91fc0cd9e0 msgr2=0x7f91fc15a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 shutdown_connections 2024-09-10T22:41:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.718+0000 7f9202cc6640 1 -- 172.21.15.66:0/3549256279 wait complete. 2024-09-10T22:41:13.730 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:41:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.835+0000 7fde4a16b640 1 -- 172.21.15.66:0/2281333994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2c003a80 msgr2=0x7fde2c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.835+0000 7fde4a16b640 1 --2- 172.21.15.66:0/2281333994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde2c003a80 0x7fde2c003ec0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fde440767f0 tx=0x7fde2c005830 comp rx=0 tx=0).stop 2024-09-10T22:41:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.835+0000 7fde43fff640 1 -- 172.21.15.66:0/2281333994 reap_dead start 2024-09-10T22:41:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.835+0000 7fde4a16b640 1 -- 172.21.15.66:0/2281333994 shutdown_connections 2024-09-10T22:41:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.835+0000 7fde4a16b640 1 -- 172.21.15.66:0/2281333994 >> 172.21.15.66:0/2281333994 conn(0x7fde440d1e30 msgr2=0x7fde440d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:13.836 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.835+0000 7fde4a16b640 1 -- 172.21.15.66:0/2281333994 shutdown_connections 2024-09-10T22:41:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.835+0000 7fde4a16b640 1 -- 172.21.15.66:0/2281333994 wait complete. 2024-09-10T22:41:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.836+0000 7fde4a16b640 1 Processor -- start 2024-09-10T22:41:13.837 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.836+0000 7fde4a16b640 1 -- start start 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde4a16b640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde2c003a80 0x7fde441f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde4a16b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde440c81c0 0x7fde440c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde4a16b640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde440b29b0 0x7fde4409e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde4a16b640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7fde440ced90 con 0x7fde440b29b0 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde4a16b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fde440cec10 con 0x7fde440c81c0 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde4a16b640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7fde440cef10 con 0x7fde2c003a80 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde440b29b0 0x7fde4409e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.839 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde440b29b0 0x7fde4409e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53078/0 (socket says 172.21.15.66:53078) 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 -- 172.21.15.66:0/2655948017 learned_addr learned my addr 172.21.15.66:0/2655948017 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde437fe640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde2c003a80 0x7fde441f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 -- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde2c003a80 msgr2=0x7fde441f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde2c003a80 0x7fde441f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 -- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde440c81c0 msgr2=0x7fde440c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde440c81c0 0x7fde440c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 -- 172.21.15.66:0/2655948017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde440c7120 con 0x7fde440b29b0 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde43fff640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde440b29b0 0x7fde4409e6d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.837+0000 7fde2bfff640 1 -- 172.21.15.66:0/2655948017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c004b20 con 0x7fde440b29b0 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.838+0000 7fde2bfff640 1 -- 172.21.15.66:0/2655948017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde3c004cc0 con 0x7fde440b29b0 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.838+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde440c7400 con 0x7fde440b29b0 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.838+0000 7fde2bfff640 1 -- 172.21.15.66:0/2655948017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c003bd0 con 0x7fde440b29b0 2024-09-10T22:41:13.840 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.838+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde440c79c0 con 0x7fde440b29b0 2024-09-10T22:41:13.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.839+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde440cc600 con 0x7fde440b29b0 2024-09-10T22:41:13.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.839+0000 7fde2bfff640 1 -- 172.21.15.66:0/2655948017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fde3c021060 con 0x7fde440b29b0 2024-09-10T22:41:13.841 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.840+0000 7fde2bfff640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fde30077fa0 0x7fde3007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:13.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.840+0000 7fde2bfff640 1 -- 172.21.15.66:0/2655948017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7fde3c099c10 con 0x7fde440b29b0 2024-09-10T22:41:13.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.840+0000 7fde437fe640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fde30077fa0 0x7fde3007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:13.842 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.840+0000 7fde437fe640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fde30077fa0 0x7fde3007a460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde34033000 tx=0x7fde440c55e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:13.845 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:13.844+0000 7fde2bfff640 1 -- 172.21.15.66:0/2655948017 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fde440cc600 con 0x7fde440b29b0 2024-09-10T22:41:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.026+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde440d06b0 con 0x7fde30077fa0 2024-09-10T22:41:14.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.027+0000 7fde2bfff640 1 -- 172.21.15.66:0/2655948017 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13410 (secure 0 0 0) 0x7fde440d06b0 con 0x7fde30077fa0 2024-09-10T22:41:14.028 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:14.028 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:41:14.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fde30077fa0 msgr2=0x7fde3007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:14.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fde30077fa0 0x7fde3007a460 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde34033000 tx=0x7fde440c55e0 comp rx=0 tx=0).stop 2024-09-10T22:41:14.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde440b29b0 msgr2=0x7fde4409e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:14.030 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde440b29b0 0x7fde4409e6d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c00f3c0 comp rx=0 tx=0).stop 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 shutdown_connections 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7fde30077fa0 0x7fde3007a460 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde440b29b0 0x7fde4409e6d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde440c81c0 0x7fde440c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 --2- 172.21.15.66:0/2655948017 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7fde2c003a80 0x7fde441f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 >> 172.21.15.66:0/2655948017 conn(0x7fde440d1e30 msgr2=0x7fde4415e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 shutdown_connections 2024-09-10T22:41:14.031 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:14.029+0000 7fde4a16b640 1 -- 172.21.15.66:0/2655948017 wait complete. 2024-09-10T22:41:14.040 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-10T22:41:12.686635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":243376,"kb_used_data":6344,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731344,"statfs":{"total":773094113280,"available":772844896256,"internally_reserved":0,"allocated":6496256,"data_stored":3397888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7290,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-74,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001473"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":175,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T22:41:05.976874+0000","last_change":"2024-09-10T22:34:51.875343+0000","last_active":"2024-09-10T22:41:05.976874+0000","last_peered":"2024-09-10T22:41:05.976874+0000","last_clean":"2024-09-10T22:41:05.976874+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:41:05.976874+0000","last_undegraded":"2024-09-10T22:41:05.976874+0000","last_fullsized":"2024-09-10T22:41:05.976874+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:34:50.792160+0000","last_clean_scrub_stamp":"2024-09-10T22:34:50.792160+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:22.286766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509519,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":867512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":867512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542225,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27676,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:41:14.044 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T22:34:50.792160+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=22, tm_min=34, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=22, tm_min=41, tm_sec=9, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T22:41:14.044 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T22:41:33.129 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:33.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:34.044 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T22:41:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.162+0000 7f769e7c9640 1 -- 172.21.15.66:0/3156405904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7680003a00 msgr2=0x7f7680003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.162+0000 7f769e7c9640 1 --2- 172.21.15.66:0/3156405904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7680003a00 0x7f7680003e20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f768800c260 tx=0x7f7680005830 comp rx=0 tx=0).stop 2024-09-10T22:41:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.162+0000 7f769cd66640 1 -- 172.21.15.66:0/3156405904 reap_dead start 2024-09-10T22:41:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.163+0000 7f769e7c9640 1 -- 172.21.15.66:0/3156405904 shutdown_connections 2024-09-10T22:41:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.163+0000 7f769e7c9640 1 -- 172.21.15.66:0/3156405904 >> 172.21.15.66:0/3156405904 conn(0x7f76980b1950 msgr2=0x7f76980b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.163+0000 7f769e7c9640 1 -- 172.21.15.66:0/3156405904 shutdown_connections 2024-09-10T22:41:34.164 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.163+0000 7f769e7c9640 1 -- 172.21.15.66:0/3156405904 wait complete. 2024-09-10T22:41:34.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.164+0000 7f769e7c9640 1 Processor -- start 2024-09-10T22:41:34.165 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.164+0000 7f769e7c9640 1 -- start start 2024-09-10T22:41:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769e7c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7680003a00 0x7f769814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769e7c9640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f769814eda0 0x7f7698144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769e7c9640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769815d890 0x7f769815dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769e7c9640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f76980cb210 con 0x7f769815d890 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769e7c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f76980cb090 con 0x7f7680003a00 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769e7c9640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f76980cb390 con 0x7f769814eda0 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769815d890 0x7f769815dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769815d890 0x7f769815dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57688/0 (socket says 172.21.15.66:57688) 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 -- 172.21.15.66:0/2105693542 learned_addr learned my addr 172.21.15.66:0/2105693542 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f76977fe640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f769814eda0 0x7f7698144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f7697fff640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7680003a00 0x7f769814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 -- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f769814eda0 msgr2=0x7f7698144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f769814eda0 0x7f7698144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 -- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7680003a00 msgr2=0x7f769814f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7680003a00 0x7f769814f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 -- 172.21.15.66:0/2105693542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76981485f0 con 0x7f769815d890 2024-09-10T22:41:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f7697fff640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7680003a00 0x7f769814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:41:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.165+0000 7f769cd66640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769815d890 0x7f769815dd40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7690011cf0 tx=0x7f769000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.166+0000 7f7694ff9640 1 -- 172.21.15.66:0/2105693542 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690003d80 con 0x7f769815d890 2024-09-10T22:41:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.166+0000 7f7694ff9640 1 -- 172.21.15.66:0/2105693542 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f769000fe80 con 0x7f769815d890 2024-09-10T22:41:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.166+0000 7f7694ff9640 1 -- 172.21.15.66:0/2105693542 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690004d80 con 0x7f769815d890 2024-09-10T22:41:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.166+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76981488d0 con 0x7f769815d890 2024-09-10T22:41:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.166+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7698148e90 con 0x7f769815d890 2024-09-10T22:41:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.167+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76980ca940 con 0x7f769815d890 2024-09-10T22:41:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.167+0000 7f7694ff9640 1 -- 172.21.15.66:0/2105693542 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7690018030 con 0x7f769815d890 2024-09-10T22:41:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.168+0000 7f7694ff9640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7684077fa0 0x7f768407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.168+0000 7f7694ff9640 1 -- 172.21.15.66:0/2105693542 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f769009a4e0 con 0x7f769815d890 2024-09-10T22:41:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.168+0000 7f7697fff640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7684077fa0 0x7f768407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.169+0000 7f7697fff640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7684077fa0 0x7f768407a460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7688033000 tx=0x7f7688001a30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:34.173 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.172+0000 7f7694ff9640 1 -- 172.21.15.66:0/2105693542 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7690063280 con 0x7f769815d890 2024-09-10T22:41:34.356 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.355+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 --> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76980708a0 con 0x7f7684077fa0 2024-09-10T22:41:34.357 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.356+0000 7f7694ff9640 1 -- 172.21.15.66:0/2105693542 <== mgr.4108 v2:172.21.15.94:6800/718501963 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7f76980708a0 con 0x7f7684077fa0 2024-09-10T22:41:34.357 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:34.358 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T22:41:34.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.358+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7684077fa0 msgr2=0x7f768407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.358+0000 7f769e7c9640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7684077fa0 0x7f768407a460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7688033000 tx=0x7f7688001a30 comp rx=0 tx=0).stop 2024-09-10T22:41:34.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.358+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769815d890 msgr2=0x7f769815dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.358+0000 7f769e7c9640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769815d890 0x7f769815dd40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7690011cf0 tx=0x7f769000f790 comp rx=0 tx=0).stop 2024-09-10T22:41:34.360 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.358+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 shutdown_connections 2024-09-10T22:41:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.359+0000 7f769e7c9640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f7684077fa0 0x7f768407a460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.359+0000 7f769e7c9640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f769815d890 0x7f769815dd40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.359+0000 7f769e7c9640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f769814eda0 0x7f7698144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.359+0000 7f769e7c9640 1 --2- 172.21.15.66:0/2105693542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7680003a00 0x7f769814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.359+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 >> 172.21.15.66:0/2105693542 conn(0x7f76980b1950 msgr2=0x7f76980cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.359+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 shutdown_connections 2024-09-10T22:41:34.361 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.359+0000 7f769e7c9640 1 -- 172.21.15.66:0/2105693542 wait complete. 2024-09-10T22:41:34.370 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-10T22:41:32.690972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":243368,"kb_used_data":6344,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731352,"statfs":{"total":773094113280,"available":772844904448,"internally_reserved":0,"allocated":6496256,"data_stored":3397888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002732"},"pg_stats":[{"pgid":"1.0","version":"56'74","reported_seq":185,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T22:41:14.149773+0000","last_change":"2024-09-10T22:41:14.149773+0000","last_active":"2024-09-10T22:41:14.149773+0000","last_peered":"2024-09-10T22:41:14.149773+0000","last_clean":"2024-09-10T22:41:14.149773+0000","last_became_active":"2024-09-10T22:34:51.874289+0000","last_became_peered":"2024-09-10T22:34:51.874289+0000","last_unstale":"2024-09-10T22:41:14.149773+0000","last_undegraded":"2024-09-10T22:41:14.149773+0000","last_fullsized":"2024-09-10T22:41:14.149773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'74","last_scrub_stamp":"2024-09-10T22:41:14.149674+0000","last_deep_scrub":"56'74","last_deep_scrub_stamp":"2024-09-10T22:41:14.149674+0000","last_clean_scrub_stamp":"2024-09-10T22:41:14.149674+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:12.621186+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509523,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":867512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":867512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542229,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27676,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":663552,"data_stored":277144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:41:34.372 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T22:41:34.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.475+0000 7f5c4695d640 1 -- 172.21.15.66:0/4219461325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d10c0 msgr2=0x7f5c40167c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.475+0000 7f5c4695d640 1 --2- 172.21.15.66:0/4219461325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d10c0 0x7f5c40167c60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5c40161b90 tx=0x7f5c3802aac0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.475+0000 7f5c4695d640 1 -- 172.21.15.66:0/4219461325 shutdown_connections 2024-09-10T22:41:34.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.475+0000 7f5c4695d640 1 --2- 172.21.15.66:0/4219461325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d10c0 0x7f5c40167c60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.477 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.475+0000 7f5c4695d640 1 --2- 172.21.15.66:0/4219461325 >> v2:172.21.15.94:3300/0 conn(0x7f5c400d2d70 0x7f5c400d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.475+0000 7f5c4695d640 1 -- 172.21.15.66:0/4219461325 >> 172.21.15.66:0/4219461325 conn(0x7f5c400d4060 msgr2=0x7f5c400d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:34.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.476+0000 7f5c4695d640 1 -- 172.21.15.66:0/4219461325 shutdown_connections 2024-09-10T22:41:34.478 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.476+0000 7f5c4695d640 1 -- 172.21.15.66:0/4219461325 wait complete. 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.477+0000 7f5c4695d640 1 Processor -- start 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.477+0000 7f5c4695d640 1 -- start start 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c4695d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c400d07a0 0x7f5c401527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c4695d640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c400d10c0 0x7f5c400cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c4695d640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d2d70 0x7f5c40150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c4695d640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40168320 con 0x7f5c400d2d70 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c4695d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401681a0 con 0x7f5c400d07a0 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c4695d640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5c401684a0 con 0x7f5c400d10c0 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d2d70 0x7f5c40150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.481 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d2d70 0x7f5c40150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57706/0 (socket says 172.21.15.66:57706) 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 -- 172.21.15.66:0/1962323868 learned_addr learned my addr 172.21.15.66:0/1962323868 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c3ffff640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c400d07a0 0x7f5c401527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c3f7fe640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c400d10c0 0x7f5c400cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 -- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c400d10c0 msgr2=0x7f5c400cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c400d10c0 0x7f5c400cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 -- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c400d07a0 msgr2=0x7f5c401527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c400d07a0 0x7f5c401527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 -- 172.21.15.66:0/1962323868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4014cbc0 con 0x7f5c400d2d70 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c3f7fe640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c400d10c0 0x7f5c400cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c44efa640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d2d70 0x7f5c40150a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c3802b0b0 tx=0x7f5c38003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38044070 con 0x7f5c400d2d70 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.478+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c38004250 con 0x7f5c400d2d70 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.479+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c380046a0 con 0x7f5c400d2d70 2024-09-10T22:41:34.482 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.479+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c400ccbb0 con 0x7f5c400d2d70 2024-09-10T22:41:34.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.479+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c400cd170 con 0x7f5c400d2d70 2024-09-10T22:41:34.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.480+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c400b6aa0 con 0x7f5c400d2d70 2024-09-10T22:41:34.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.480+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5c38036400 con 0x7f5c400d2d70 2024-09-10T22:41:34.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.481+0000 7f5c3cff9640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c2c077f70 0x7f5c2c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.483 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.481+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f5c380ba0c0 con 0x7f5c400d2d70 2024-09-10T22:41:34.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.481+0000 7f5c3ffff640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c2c077f70 0x7f5c2c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.484 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.481+0000 7f5c3ffff640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c2c077f70 0x7f5c2c07a430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5c30000f60 tx=0x7f5c30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:34.485 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.485+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5c400b6aa0 con 0x7f5c400d2d70 2024-09-10T22:41:34.669 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.668+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5c400d0b80 con 0x7f5c400d2d70 2024-09-10T22:41:34.672 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.671+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) ==== 123+0+0 (secure 0 0 0) 0x7f5c4014cbc0 con 0x7f5c400d2d70 2024-09-10T22:41:34.674 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.673+0000 7f5c3cff9640 1 -- 172.21.15.66:0/1962323868 <== mon.0 v2:172.21.15.66:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f5c38004840 con 0x7f5c400d2d70 2024-09-10T22:41:34.675 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c2c077f70 msgr2=0x7f5c2c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c2c077f70 0x7f5c2c07a430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5c30000f60 tx=0x7f5c30009040 comp rx=0 tx=0).stop 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d2d70 msgr2=0x7f5c40150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d2d70 0x7f5c40150a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c3802b0b0 tx=0x7f5c38003ce0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 shutdown_connections 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5c2c077f70 0x7f5c2c07a430 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c400d2d70 0x7f5c40150a90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5c400d10c0 0x7f5c400cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 --2- 172.21.15.66:0/1962323868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5c400d07a0 0x7f5c401527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 >> 172.21.15.66:0/1962323868 conn(0x7f5c400d4060 msgr2=0x7f5c40156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 shutdown_connections 2024-09-10T22:41:34.676 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.674+0000 7f5c4695d640 1 -- 172.21.15.66:0/1962323868 wait complete. 2024-09-10T22:41:34.687 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:41:34.687 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T22:41:34.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.792+0000 7f5e521ad640 1 -- 172.21.15.66:0/1835780460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e34004d40 msgr2=0x7f5e340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.792+0000 7f5e521ad640 1 --2- 172.21.15.66:0/1835780460 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e34004d40 0x7f5e340051c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009f00 tx=0x7f5e34005830 comp rx=0 tx=0).stop 2024-09-10T22:41:34.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.792+0000 7f5e4bfff640 1 -- 172.21.15.66:0/1835780460 reap_dead start 2024-09-10T22:41:34.793 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.792+0000 7f5e521ad640 1 -- 172.21.15.66:0/1835780460 shutdown_connections 2024-09-10T22:41:34.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.792+0000 7f5e521ad640 1 -- 172.21.15.66:0/1835780460 >> 172.21.15.66:0/1835780460 conn(0x7f5e4c0cd9e0 msgr2=0x7f5e4c0cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:34.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.792+0000 7f5e521ad640 1 -- 172.21.15.66:0/1835780460 shutdown_connections 2024-09-10T22:41:34.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.792+0000 7f5e521ad640 1 -- 172.21.15.66:0/1835780460 wait complete. 2024-09-10T22:41:34.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.793+0000 7f5e521ad640 1 Processor -- start 2024-09-10T22:41:34.794 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.793+0000 7f5e521ad640 1 -- start start 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e521ad640 1 --2- >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e4c152980 0x7f5e4c1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e521ad640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e4c0b3150 0x7f5e4c1f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e521ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e4c1f8e20 0x7f5e4c1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e521ad640 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0cb940 con 0x7f5e4c0b3150 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e521ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0cb7c0 con 0x7f5e4c1f8e20 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e521ad640 1 -- --> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c0ceb30 con 0x7f5e4c152980 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e4c0b3150 0x7f5e4c1f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e4c0b3150 0x7f5e4c1f88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57730/0 (socket says 172.21.15.66:57730) 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 -- 172.21.15.66:0/1435397523 learned_addr learned my addr 172.21.15.66:0/1435397523 (peer_addr_for_me v2:172.21.15.66:0/0) 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4b7fe640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e4c152980 0x7f5e4c1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4bfff640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e4c1f8e20 0x7f5e4c1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.797 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 -- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e4c152980 msgr2=0x7f5e4c1f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e4c152980 0x7f5e4c1f83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 -- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e4c1f8e20 msgr2=0x7f5e4c1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e4c1f8e20 0x7f5e4c1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 -- 172.21.15.66:0/1435397523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c0b36c0 con 0x7f5e4c0b3150 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4bfff640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e4c1f8e20 0x7f5e4c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e4affd640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e4c0b3150 0x7f5e4c1f88e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e40006680 tx=0x7f5e400066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.794+0000 7f5e33fff640 1 -- 172.21.15.66:0/1435397523 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40015020 con 0x7f5e4c0b3150 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.795+0000 7f5e33fff640 1 -- 172.21.15.66:0/1435397523 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f5e40011a30 con 0x7f5e4c0b3150 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.795+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c0b0b40 con 0x7f5e4c0b3150 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.795+0000 7f5e33fff640 1 -- 172.21.15.66:0/1435397523 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e400103c0 con 0x7f5e4c0b3150 2024-09-10T22:41:34.798 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.795+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c151180 con 0x7f5e4c0b3150 2024-09-10T22:41:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.796+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4c0b0960 con 0x7f5e4c0b3150 2024-09-10T22:41:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.796+0000 7f5e33fff640 1 -- 172.21.15.66:0/1435397523 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5e40010560 con 0x7f5e4c0b3150 2024-09-10T22:41:34.799 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.797+0000 7f5e33fff640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e38077f70 0x7f5e3807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:41:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.797+0000 7f5e33fff640 1 -- 172.21.15.66:0/1435397523 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5029+0+0 (secure 0 0 0) 0x7f5e4009aae0 con 0x7f5e4c0b3150 2024-09-10T22:41:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.797+0000 7f5e4b7fe640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e38077f70 0x7f5e3807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:41:34.800 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.798+0000 7f5e4b7fe640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e38077f70 0x7f5e3807a430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009c50 tx=0x7f5e3c005f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:41:34.802 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:34.801+0000 7f5e33fff640 1 -- 172.21.15.66:0/1435397523 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5e4c0b0960 con 0x7f5e4c0b3150 2024-09-10T22:41:35.024 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.023+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e4c0cccd0 con 0x7f5e4c0b3150 2024-09-10T22:41:35.025 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.023+0000 7f5e33fff640 1 -- 172.21.15.66:0/1435397523 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+3905 (secure 0 0 0) 0x7f5e4001f020 con 0x7f5e4c0b3150 2024-09-10T22:41:35.025 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T22:41:35.025 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":41,"btime":"2024-09-10T22:41:03:960747+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5302,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/1430083245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":1430083245},{"type":"v1","addr":"172.21.15.94:6835","nonce":1430083245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":5305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/226160603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":226160603},{"type":"v1","addr":"172.21.15.94:6837","nonce":226160603}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5314,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/815095488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6833","nonce":815095488},{"type":"v1","addr":"172.21.15.66:6835","nonce":815095488}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6834/1161461705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":1161461705},{"type":"v1","addr":"172.21.15.66:6834","nonce":1161461705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[]} 2024-09-10T22:41:35.026 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 41 2024-09-10T22:41:35.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.025+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e38077f70 msgr2=0x7f5e3807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:35.027 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.025+0000 7f5e521ad640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e38077f70 0x7f5e3807a430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009c50 tx=0x7f5e3c005f90 comp rx=0 tx=0).stop 2024-09-10T22:41:35.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.025+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e4c0b3150 msgr2=0x7f5e4c1f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:41:35.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e4c0b3150 0x7f5e4c1f88e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e40006680 tx=0x7f5e400066b0 comp rx=0 tx=0).stop 2024-09-10T22:41:35.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 shutdown_connections 2024-09-10T22:41:35.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:6800/718501963,v1:172.21.15.94:6801/718501963] conn(0x7f5e38077f70 0x7f5e3807a430 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:35.028 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e4c1f8e20 0x7f5e4c1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:35.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e4c0b3150 0x7f5e4c1f88e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:35.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 --2- 172.21.15.66:0/1435397523 >> [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] conn(0x7f5e4c152980 0x7f5e4c1f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:41:35.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 >> 172.21.15.66:0/1435397523 conn(0x7f5e4c0cd9e0 msgr2=0x7f5e4c071a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:41:35.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 shutdown_connections 2024-09-10T22:41:35.029 INFO:teuthology.orchestra.run.smithi066.stderr:2024-09-10T22:41:35.026+0000 7f5e521ad640 1 -- 172.21.15.66:0/1435397523 wait complete. 2024-09-10T22:41:35.038 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T22:41:35.038 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T22:41:35.038 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T22:41:35.038 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.047 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T22:41:35.047 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T22:41:35.047 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.096 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T22:41:35.096 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T22:41:35.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.108 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T22:41:35.108 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T22:41:35.108 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.159 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T22:41:35.159 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T22:41:35.160 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T22:41:35.160 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.186 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T22:41:35.187 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T22:41:35.187 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.254 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T22:41:35.254 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T22:41:35.254 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.331 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T22:41:35.332 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T22:41:35.332 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.397 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T22:41:35.397 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T22:41:35.397 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.433 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T22:41:35.434 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T22:41:35.434 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.501 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T22:41:35.501 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T22:41:35.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.577 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T22:41:35.577 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T22:41:35.577 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.653 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T22:41:35.653 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T22:41:35.654 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T22:41:35.654 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.680 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T22:41:35.680 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T22:41:35.681 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.724 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T22:41:35.724 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T22:41:35.724 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T22:41:35.725 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.725 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-10T22:41:35.736 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T22:41:35.737 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T22:41:35.737 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.777 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T22:41:35.777 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T22:41:35.777 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:41:35.830 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T22:41:35.830 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T22:41:35.831 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-10T22:41:35.864 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:41:35.865 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T22:41:36.122 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:41:36.122 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T22:41:36.228 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:41:36.228 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T22:41:36.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:41:36.279 DEBUG:teuthology.orchestra.run.smithi066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T22:41:36.328 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:41:36.328 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T22:41:36.503 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:41:36.503 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T22:41:36.574 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:41:36.574 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T22:41:36.631 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:41:36.631 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T22:41:36.700 INFO:tasks.ceph:Archiving mon data... 2024-09-10T22:41:36.701 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/data/mon.a.tgz 2024-09-10T22:41:36.701 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-09-10T22:41:36.717 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.faZBvv8wXa 2024-09-10T22:41:36.718 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.faZBvv8wXa 2024-09-10T22:41:36.877 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.faZBvv8wXa 2024-09-10T22:41:36.962 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.faZBvv8wXa is 396KB 2024-09-10T22:41:37.059 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.faZBvv8wXa 2024-09-10T22:41:37.075 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/data/mon.b.tgz 2024-09-10T22:41:37.075 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-10T22:41:37.093 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.pu2DBiJCbd 2024-09-10T22:41:37.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.pu2DBiJCbd 2024-09-10T22:41:37.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.pu2DBiJCbd 2024-09-10T22:41:37.360 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.pu2DBiJCbd is 524KB 2024-09-10T22:41:37.432 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.pu2DBiJCbd 2024-09-10T22:41:37.447 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/data/mon.c.tgz 2024-09-10T22:41:37.448 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-10T22:41:37.502 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.AxBxVNQOUC 2024-09-10T22:41:37.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.AxBxVNQOUC 2024-09-10T22:41:37.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.AxBxVNQOUC 2024-09-10T22:41:37.754 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.AxBxVNQOUC is 524KB 2024-09-10T22:41:37.826 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.AxBxVNQOUC 2024-09-10T22:41:37.841 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T22:41:37.841 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T22:41:37.843 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T22:41:37.926 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T22:41:37.963 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T22:41:37.963 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T22:41:37.965 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/remote/smithi066/crash 2024-09-10T22:41:37.966 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T22:41:37.994 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/remote/smithi094/crash 2024-09-10T22:41:37.995 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T22:41:38.028 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T22:41:38.028 DEBUG:teuthology.orchestra.run.smithi066:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:38.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:38.128 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:38.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T22:41:38.179 INFO:tasks.ceph:Compressing logs... 2024-09-10T22:41:38.179 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T22:41:38.197 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T22:41:38.264 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42083.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36547.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.1.42083.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42083.log.gz 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36547.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36547.log.gz 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36385.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36507.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36587.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36385.log.gz 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36507.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36507.log.gz 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log/var/log/ceph/ceph-client.admin.36587.log: 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.4.log: 86.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.36587.log.gz 2024-09-10T22:41:38.265 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42481.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36467.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.3.42481.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.3.42481.log.gz 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36427.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36467.log: gzip 93.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36467.log.gz 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36427.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36427.log.gz 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.5.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36667.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36627.log 2024-09-10T22:41:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36667.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T22:41:38.267 INFO:teuthology.orchestra.run.smithi094.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36667.log.gz 2024-09-10T22:41:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.36627.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T22:41:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mds.d.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36627.log.gz 2024-09-10T22:41:38.267 INFO:teuthology.orchestra.run.smithi094.stderr: 92.5%gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T22:41:38.267 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T22:41:38.267 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.b.log: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36678.log 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50834.log 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36516.log: /var/log/ceph/ceph-client.admin.36678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44618.log 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.36678.log.gz 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2024-09-10T22:41:38.335 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47383.log 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44618.log: /var/log/ceph/ceph-client.admin.50270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50228.log 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47383.log: 90.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.44618.log.gz -5 --verbose 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.43451.log 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47383.log.gz 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50228.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50228.log.gz 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47860.log 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47073.log 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53914.log 2024-09-10T22:41:38.336 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47860.log.gz 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45848.log 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47073.log.gz 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46295.log 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53914.log.gz 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53050.log 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45848.log.gz 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47427.log 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46295.log.gz 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43274.log 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53050.log.gz 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47427.log.gz 2024-09-10T22:41:38.337 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47031.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43274.log.gz 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36685.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46432.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47031.log.gz 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48156.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36685.log.gz 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46432.log.gz 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.48156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46642.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48156.log.gz 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53356.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47328.log: /var/log/ceph/ceph-client.admin.46642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48381.log 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz 2024-09-10T22:41:38.338 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46642.log.gz 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53356.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53356.log.gz 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44576.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43792.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.48381.log.gz 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48522.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44576.log.gz 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51048.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43792.log: /var/log/ceph/ceph-client.admin.48522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45242.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43792.log.gz 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48522.log.gz 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36446.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51048.log.gz 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43791.log 2024-09-10T22:41:38.339 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45242.log: /var/log/ceph/ceph-client.admin.36446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44008.log 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45242.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36446.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54124.log 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43791.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47734.log 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44008.log: /var/log/ceph/ceph-client.admin.54124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52677.log 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.44008.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54124.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46558.log 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47734.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51989.log 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54040.log 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46558.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52677.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46558.log.gz 2024-09-10T22:41:38.340 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49972.log 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51989.log.gz 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46516.log 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54040.log: /var/log/ceph/ceph-client.admin.49972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48564.log 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54040.log.gz 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49972.log.gz 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52965.log 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46516.log.gz 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.48564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54208.log 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44366.log 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52965.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48564.log.gz 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54208.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.51881.log 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52965.log.gz 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.54208.log.gz 2024-09-10T22:41:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44127.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44366.log.gz 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50652.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gz 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44127.log: /var/log/ceph/ceph-client.admin.50652.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44127.log.gz 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36279.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50652.log.gz 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49827.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47286.log: /var/log/ceph/ceph-client.admin.36279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43064.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr: 76.3% -- replaced with /var/log/ceph/ceph-client.admin.36279.log.gz 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43786.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49827.log.gz 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46829.log 2024-09-10T22:41:38.342 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43064.log.gz 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50084.log 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43786.log: /var/log/ceph/ceph-client.admin.46829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46600.log 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43786.log.gz 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46829.log.gz 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43022.log 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.50084.log.gz --verbose 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.46744.log 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46600.log: /var/log/ceph/ceph-client.admin.43022.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46600.log.gz 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47608.log 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43022.log.gz 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51737.log 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46744.log: /var/log/ceph/ceph-client.admin.47608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53872.log 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46744.log.gz 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47608.log.gz 2024-09-10T22:41:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47159.log 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53872.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.51737.log.gz 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.50783.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52302.log 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53872.log.gz 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47159.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54351.log 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47159.log.gz 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43456.log 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52302.log.gz 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36307.log 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54351.log: /var/log/ceph/ceph-client.admin.43456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36723.log 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54351.log.gz 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43456.log.gz 2024-09-10T22:41:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36307.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36307.log.gz 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36723.log: /var/log/ceph/ceph.audit.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36723.log.gz 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45806.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43106.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43190.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45806.log.gz 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52426.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36716.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45680.log 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43106.log.gz 2024-09-10T22:41:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36716.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52426.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42221.log 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36716.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37270.log 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45680.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42441.log 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42221.log: /var/log/ceph/ceph-client.admin.37270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49869.log 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42221.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37270.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53008.log 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42441.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45764.log 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49869.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49869.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48301.log 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53008.log.gz 2024-09-10T22:41:38.346 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49659.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45764.log: /var/log/ceph/ceph-client.admin.48301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43789.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45764.log.gz 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48301.log.gz 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50014.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49659.log.gz 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47818.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43789.log: /var/log/ceph/ceph-client.admin.50014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53746.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43789.log.gz 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50014.log.gz 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54269.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47818.log.gz 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53998.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53746.log: /var/log/ceph/ceph-client.admin.54269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53482.log 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53746.log.gz 2024-09-10T22:41:38.347 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54269.log.gz 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53120.log 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45546.log 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53998.log.gz 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53482.log: /var/log/ceph/ceph-client.admin.53120.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53482.log.gz 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42938.log 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53120.log.gz 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42770.log 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.2.45546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45546.log.gz 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52761.log 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.42938.log.gz 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44126.log 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42770.log.gz 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44450.log 2024-09-10T22:41:38.348 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52761.log.gz 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43452.log 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44126.log: /var/log/ceph/ceph-client.admin.44450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47902.log 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44126.log.gz 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44450.log.gz 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43452.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43452.log.gz 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47902.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51006.log 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47902.log.gz 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50312.log 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44756.log 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51006.log: /var/log/ceph/ceph-client.admin.50312.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51006.log.gz 2024-09-10T22:41:38.349 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49701.log 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50312.log.gz 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44756.log.gz 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47692.log 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49701.log: /var/log/ceph/ceph-client.admin.44128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53620.log 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49701.log.gz 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42682.log 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47692.log.gz 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46684.log 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53620.log.gz 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 87.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42682.log.gz 2024-09-10T22:41:38.350 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.45722.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46684.log: /var/log/ceph/ceph-client.admin.46989.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.46684.log.gz 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47562.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42896.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.47562.log: -5 --verbose 87.9% -- -- replaced with /var/log/ceph/ceph-client.admin.45722.log.gz /var/log/ceph/ceph-client.admin.42399.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47562.log.gz 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47968.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42896.log.gz 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45932.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42399.log.gz 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47968.log: /var/log/ceph/ceph-client.admin.45932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-09-10T22:41:38.351 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47968.log.gz 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45932.log.gz 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47243.log: /var/log/ceph/ceph-client.admin.42589.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz /var/log/ceph/ceph-client.admin.36719.log 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53535.log 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36719.log.gz 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51090.log 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53535.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T22:41:38.352 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53535.log.gz 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52845.log 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51090.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47776.log 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51090.log.gz 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42812.log 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52845.log.gz 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36363.log 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47776.log.gz 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54082.log 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36363.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42812.log.gz 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51132.log 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.36363.log.gz 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54082.log: /var/log/ceph/ceph-client.admin.51132.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54082.log.gz 2024-09-10T22:41:38.353 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53398.log 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4%gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43148.log: /var/log/ceph/ceph-client.admin.44123.log 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51132.log.gz 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52887.log 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53398.log.gz 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44123.log: /var/log/ceph/ceph-client.admin.52887.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.44123.log.gz --verbose 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr: -- /var/log/ceph/ceph-client.admin.50918.log 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52887.log.gz 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49413.log 2024-09-10T22:41:38.354 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52803.log 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50918.log: /var/log/ceph/ceph-client.admin.49413.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50918.log.gz 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52803.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49413.log.gz 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52635.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52803.log.gz 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53830.log 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gz 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46127.log 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr: 87.0%/var/log/ceph/ceph-client.admin.53830.log: -- replaced with /var/log/ceph/ceph-client.admin.52635.log.gz 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51657.log 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46127.log: gzip -5 --verbose -- 87.0% /var/log/ceph/ceph-client.admin.42980.log 2024-09-10T22:41:38.355 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53830.log.gz 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46205.log 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46127.log.gz 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51657.log.gz 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36969.log 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42980.log: /var/log/ceph/ceph-client.admin.46205.log: gzip -5 --verbose 87.1% -- /var/log/ceph/ceph-client.admin.44408.log 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46205.log.gz 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50694.log 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36969.log.gz 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49785.log 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44492.log 2024-09-10T22:41:38.356 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50694.log.gz 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49785.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6%/var/log/ceph/ceph-client.admin.44492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49743.log 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49785.log.gz 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44492.log.gz 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52178.log/var/log/ceph/ceph-client.admin.49743.log: 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50186.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49743.log.gz 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44125.log 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43767.log 2024-09-10T22:41:38.357 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52178.log.gz/var/log/ceph/ceph-client.admin.44125.log: 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44125.log.gz 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.46474.log 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45974.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43767.log.gz 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46474.log: gzip -5 87.4% --verbose -- /var/log/ceph/ceph.log 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46474.log.gz 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51512.log 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45974.log.gz 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51216.log 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43784.log 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.0.51512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51512.log.gz 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36604.log 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51216.log.gz 2024-09-10T22:41:38.358 INFO:teuthology.orchestra.run.smithi066.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43784.log: gzip -5 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43784.log.gz 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53162.log 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36604.log.gz 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36243.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50529.log 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53162.log.gz/var/log/ceph/ceph-client.admin.50529.log: 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46909.log 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50529.log.gz 2024-09-10T22:41:38.359 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42140.log 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51258.log: /var/log/ceph/ceph-client.admin.46909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53956.log 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42140.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46909.log.gz 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43448.log 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42140.log.gz 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51174.log 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:gzip 87.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43448.log: -- replaced with /var/log/ceph/ceph-client.admin.53956.log.gz 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51174.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43448.log.gz 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45106.log 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51174.log.gz 2024-09-10T22:41:38.360 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49617.log: /var/log/ceph/ceph-client.0.45106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53704.log 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45106.log.gz 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36335.log 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz/var/log/ceph/ceph-client.admin.53704.log: 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2%/var/log/ceph/ceph-client.admin.36335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47520.log 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53704.log.gz 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.36335.log.gz 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49925.log 2024-09-10T22:41:38.361 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:gzip/var/log/ceph/ceph-client.admin.47520.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44324.log 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49925.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47520.log.gz 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49925.log.gz 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr: /var/log/ceph/ceph-client.admin.43450.log 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.44324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43790.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44324.log.gz 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53440.log 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43450.log.gz 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:gzip 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43790.log.gz 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53440.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36189.log 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43454.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53440.log.gz 2024-09-10T22:41:38.362 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36496.log 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz 2024-09-10T22:41:38.363 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.363 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36189.log: gzip 77.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.50610.log 2024-09-10T22:41:38.363 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36189.log.gz 2024-09-10T22:41:38.363 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50964.log 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36496.log.gz 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43455.log 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50126.log 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50964.log: -- replaced with /var/log/ceph/ceph-client.admin.50610.log.gz 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43455.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50964.log.gz 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43455.log.gz 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49523.log 2024-09-10T22:41:38.364 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50126.log: /var/log/ceph/ceph-client.admin.43447.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50126.log.gz 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52719.log 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.49523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50739.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49523.log.gz 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53662.log 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52719.log.gz 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50449.log 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53662.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50739.log.gz 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52541.log 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53662.log.gz 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52116.log 2024-09-10T22:41:38.365 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50449.log.gz 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52364.log 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52541.log.gz 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.52116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52116.log.gz 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5/var/log/ceph/ceph-client.admin.52364.log: --verbose -- /var/log/ceph/ceph-client.admin.54309.log 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46787.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52364.log.gz 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50876.log 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42854.log 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50876.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54309.log.gz 2024-09-10T22:41:38.366 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50354.log 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50876.log.gz 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36745.log 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42854.log.gz 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.50354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45890.log 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50354.log.gz 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52240.log 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.36745.log.gz 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46016.log 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.45890.log: /var/log/ceph/ceph-client.admin.52240.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36657.log 2024-09-10T22:41:38.367 INFO:teuthology.orchestra.run.smithi066.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45890.log.gz 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52240.log.gz 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53577.log 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46016.log.gz 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.36657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42724.log 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36657.log.gz 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53577.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30140.log 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53577.log.gz 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42724.log.gz 2024-09-10T22:41:38.368 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph.tmp-client.admin.30140.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30140.log.gz 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51923.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54166.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.51923.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.54166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53788.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51923.log.gz 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54166.log.gz 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47478.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.53788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43785.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53788.log.gz 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-09-10T22:41:38.369 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.43785.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47478.log.gz 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43785.log.gz 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42483.log 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47650.log 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.42483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46390.log 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.47650.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42483.log.gz 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47650.log.gz 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.46390.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46390.log.gz 2024-09-10T22:41:38.370 INFO:teuthology.orchestra.run.smithi094.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T22:41:38.387 INFO:teuthology.orchestra.run.smithi094.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T22:41:38.433 INFO:teuthology.orchestra.run.smithi094.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T22:41:38.436 INFO:teuthology.orchestra.run.smithi066.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T22:41:38.485 INFO:teuthology.orchestra.run.smithi094.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T22:41:38.519 INFO:teuthology.orchestra.run.smithi066.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T22:41:38.522 INFO:teuthology.orchestra.run.smithi066.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T22:41:38.525 INFO:teuthology.orchestra.run.smithi094.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T22:41:38.526 INFO:teuthology.orchestra.run.smithi066.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T22:41:38.534 INFO:teuthology.orchestra.run.smithi094.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T22:41:38.538 INFO:teuthology.orchestra.run.smithi094.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T22:41:38.540 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-10T22:41:38.540 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m0.306s 2024-09-10T22:41:38.540 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m1.472s 2024-09-10T22:41:38.540 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.075s 2024-09-10T22:41:38.576 INFO:teuthology.orchestra.run.smithi066.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T22:41:38.615 INFO:teuthology.orchestra.run.smithi066.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T22:41:38.617 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T22:41:38.617 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m0.407s 2024-09-10T22:41:38.617 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m1.664s 2024-09-10T22:41:38.617 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.220s 2024-09-10T22:41:38.618 INFO:tasks.ceph:Archiving logs... 2024-09-10T22:41:38.618 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/remote/smithi066/log 2024-09-10T22:41:38.619 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:41:39.071 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/remote/smithi094/log 2024-09-10T22:41:39.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:41:39.282 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T22:41:39.302 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T22:41:39.302 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:41:39.305 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:41:39.397 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T22:41:39.406 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T22:41:39.406 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:41:39.409 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:41:39.422 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:41:39.424 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:41:39.427 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:41:39.427 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-09-10T22:41:39.427 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 -137us[ -113us] +/- 57ms 2024-09-10T22:41:39.427 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 +690us[ +690us] +/- 55ms 2024-09-10T22:41:39.427 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -772us[ -772us] +/- 85ms 2024-09-10T22:41:39.427 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:41:39.428 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:41:39.429 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-09-10T22:41:39.429 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 -116us[ -108us] +/- 57ms 2024-09-10T22:41:39.429 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 +716us[ +716us] +/- 55ms 2024-09-10T22:41:39.429 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -752us[ -743us] +/- 85ms 2024-09-10T22:41:39.429 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:41:39.429 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T22:41:39.437 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T22:41:39.438 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T22:41:39.446 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:41:39.540 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:41:39.605 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T22:41:39.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:41:39.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T22:41:39.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:41:39.662 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T22:41:39.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T22:41:39.680 INFO:teuthology.task.internal:Duration was 790.709600 seconds 2024-09-10T22:41:39.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T22:41:39.689 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T22:41:39.690 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:41:39.692 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:41:39.730 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:41:39.742 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:41:40.019 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T22:41:40.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T22:41:40.020 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:41:40.081 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-09-10T22:41:40.082 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:41:40.109 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T22:41:40.110 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:41:40.124 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:41:40.175 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T22:41:40.176 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:41:40.191 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:41:40.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T22:41:40.330 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T22:41:40.330 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:41:40.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:41:40.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T22:41:40.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:41:40.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:41:40.421 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-09-10T22:41:40.445 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-09-10T22:41:40.458 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:41:40.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:41:40.507 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:41:40.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:41:40.522 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T22:41:40.532 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T22:41:40.532 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/remote/smithi066 2024-09-10T22:41:40.533 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:41:40.582 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899736/remote/smithi094 2024-09-10T22:41:40.582 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:41:40.618 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T22:41:40.618 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:41:40.620 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:41:40.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T22:41:40.679 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T22:41:40.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T22:41:40.688 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T22:41:40.689 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:41:40.691 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:41:40.704 INFO:teuthology.orchestra.run.smithi066.stdout: 266064 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:41 /home/ubuntu/cephtest 2024-09-10T22:41:40.723 INFO:teuthology.orchestra.run.smithi094.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:41 /home/ubuntu/cephtest 2024-09-10T22:41:40.724 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T22:41:40.735 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T22:41:40.758 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/openfiletable} duration: 790.7095999717712 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T22:41:40.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:41:40.769 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-10T22:41:40.855 INFO:teuthology.run:pass